[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:19 dist-test-slave-xt4k krb5kdc[12361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:19 dist-test-slave-xt4k krb5kdc[12361](info): set up 2 sockets
Aug 06 04:34:19 dist-test-slave-xt4k krb5kdc[12361](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 04:34:20.248760 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:20 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:20.376118 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40883
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:40883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:20.490572 12378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:20.490831 12378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:20.490870 12378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:20.494799 12378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:20.494849 12378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:20.494869 12378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:20.494884 12378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:20.494897 12378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:20.500011 12378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:40883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:20.501031 12378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:20.502030 12378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:34:20.511349 12378 server_base.cc:1034] running on GCE node
W20260806 04:34:20.511421 12383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:20.511284 12387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:20.511292 12385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:20.512182 12378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:20.513454 12378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:20.514629 12378 hybrid_clock.cc:648] HybridClock initialized: now 1785990860514614 us; error 30 us; skew 500 ppm
Aug 06 04:34:20 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990860, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:20.517611 12378 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:20.518572 12378 webserver.cc:460] Webserver started at http://127.12.15.254:39835/ using document root <none> and password file <none>
I20260806 04:34:20.519263 12378 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:20.519340 12378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:20.519598 12378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:20.521895 12378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "496d970cf9204d86a3858160440ccfcc"
format_stamp: "Formatted at 2026-08-06 04:34:20 on dist-test-slave-xt4k"
server_key: "8dad272bd8b5967c9605ec06bd15821c"
server_key_iv: "fb8e089267bee1e3159e62570cb72280"
server_key_version: "encryptionkey@0"
I20260806 04:34:20.522539 12378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "496d970cf9204d86a3858160440ccfcc"
format_stamp: "Formatted at 2026-08-06 04:34:20 on dist-test-slave-xt4k"
server_key: "8dad272bd8b5967c9605ec06bd15821c"
server_key_iv: "fb8e089267bee1e3159e62570cb72280"
server_key_version: "encryptionkey@0"
I20260806 04:34:20.526618 12378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:20.529342 12398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:20.530647 12378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:20.530761 12378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "496d970cf9204d86a3858160440ccfcc"
format_stamp: "Formatted at 2026-08-06 04:34:20 on dist-test-slave-xt4k"
server_key: "8dad272bd8b5967c9605ec06bd15821c"
server_key_iv: "fb8e089267bee1e3159e62570cb72280"
server_key_version: "encryptionkey@0"
I20260806 04:34:20.530911 12378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:20.549537 12378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:20.552996 12378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:20.553195 12378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:20.565129 12378 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:40883
I20260806 04:34:20.565140 12472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:40883 every 8 connection(s)
I20260806 04:34:20.566200 12378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:20.569692 12473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:20.573652 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12378
I20260806 04:34:20.573786 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:20.575145 12351 external_mini_cluster.cc:1428] Setting key a7870d01f29fbc56bc2fc62c973fa836
I20260806 04:34:20.576872 12473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc: Bootstrap starting.
I20260806 04:34:20.581015 12473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:20.582371 12473 log.cc:824] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:20.584628 12473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc: No bootstrap required, opened a new log
I20260806 04:34:20.589187 12473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496d970cf9204d86a3858160440ccfcc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40883 } }
I20260806 04:34:20.589425 12473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:20.589500 12473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496d970cf9204d86a3858160440ccfcc, State: Initialized, Role: FOLLOWER
I20260806 04:34:20.590037 12473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496d970cf9204d86a3858160440ccfcc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40883 } }
I20260806 04:34:20.590225 12473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:20.590312 12473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:20.590575 12473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:20.591951 12473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496d970cf9204d86a3858160440ccfcc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40883 } }
I20260806 04:34:20.592453 12473 leader_election.cc:302] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 496d970cf9204d86a3858160440ccfcc; no voters:
I20260806 04:34:20.592859 12473 leader_election.cc:288] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:20.593039 12479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:20.593394 12479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [term 1 LEADER]: Becoming Leader. State: Replica: 496d970cf9204d86a3858160440ccfcc, State: Running, Role: LEADER
I20260806 04:34:20.593813 12479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496d970cf9204d86a3858160440ccfcc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40883 } }
I20260806 04:34:20.594385 12473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:20.595963 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 496d970cf9204d86a3858160440ccfcc. Latest consensus state: current_term: 1 leader_uuid: "496d970cf9204d86a3858160440ccfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496d970cf9204d86a3858160440ccfcc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40883 } } }
I20260806 04:34:20.596115 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:20.596385 12481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "496d970cf9204d86a3858160440ccfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496d970cf9204d86a3858160440ccfcc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40883 } } }
I20260806 04:34:20.596479 12481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc [sys.catalog]: This master's current role is: LEADER
Aug 06 04:34:20 dist-test-slave-xt4k krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:20.596860 12492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:20.600746 12492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:20.604094 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:20.578414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.12.15.254:40883)
0806 04:34:20.579251 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:20.579304 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:20.580261 (+ 957us) server_negotiation.cc:407] Connection header received
0806 04:34:20.585151 (+ 4890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:20.585171 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:20.585512 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:20.585829 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:20.588552 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:20.589687 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:20.591631 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:20.592185 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:20.592316 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:20.597770 (+ 5454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:20.597785 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:20.597802 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:20.597837 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:20.599897 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:20.600653 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:20.600660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:20.600666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:20.600830 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:20.601216 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:20.601219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:20.601221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:20.601821 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:20.602029 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:20.602816 (+ 787us) server_negotiation.cc:299] Negotiation successful
0806 04:34:20.603130 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":320,"threads_started":1}
I20260806 04:34:20.607331 12492 catalog_manager.cc:1347] Generated new cluster ID: 1f2891b45e824730a872bc978fe12200
I20260806 04:34:20.607398 12492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:20.620828 12492 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:20.621812 12492 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:20.635344 12492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 496d970cf9204d86a3858160440ccfcc: Generated new TSK 0
I20260806 04:34:20.636163 12492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:20.675045 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40883
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:20.807668 12508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:20.807906 12508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:20.807955 12508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:20.811949 12508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:20.812006 12508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:20.812065 12508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:20.817207 12508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:20.818156 12508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:20.819096 12508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:20.829555 12514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:20.829560 12518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:20.829639 12508 server_base.cc:1034] running on GCE node
W20260806 04:34:20.829792 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:20.830160 12508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:20.830761 12508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:20.831924 12508 hybrid_clock.cc:648] HybridClock initialized: now 1785990860831910 us; error 27 us; skew 500 ppm
Aug 06 04:34:20 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990860, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:20.834662 12508 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:20.835551 12508 webserver.cc:460] Webserver started at http://127.12.15.193:44193/ using document root <none> and password file <none>
I20260806 04:34:20.836095 12508 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:20.836161 12508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:20.836397 12508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:20.838569 12508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "b5a4aabbac25472da2201de838becc06"
format_stamp: "Formatted at 2026-08-06 04:34:20 on dist-test-slave-xt4k"
server_key: "ab74e04695d0a87c5900de2d03e41ea7"
server_key_iv: "5c4db7c811abb90c669cdb751a6096ff"
server_key_version: "encryptionkey@0"
I20260806 04:34:20.839116 12508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "b5a4aabbac25472da2201de838becc06"
format_stamp: "Formatted at 2026-08-06 04:34:20 on dist-test-slave-xt4k"
server_key: "ab74e04695d0a87c5900de2d03e41ea7"
server_key_iv: "5c4db7c811abb90c669cdb751a6096ff"
server_key_version: "encryptionkey@0"
I20260806 04:34:20.842826 12508 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 04:34:20.845230 12531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:20.846557 12508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 04:34:20.846673 12508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "b5a4aabbac25472da2201de838becc06"
format_stamp: "Formatted at 2026-08-06 04:34:20 on dist-test-slave-xt4k"
server_key: "ab74e04695d0a87c5900de2d03e41ea7"
server_key_iv: "5c4db7c811abb90c669cdb751a6096ff"
server_key_version: "encryptionkey@0"
I20260806 04:34:20.846771 12508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:20.861297 12508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:20.864250 12508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:20.864449 12508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:20.865017 12508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:20.865885 12508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:20.865931 12508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:20.865970 12508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:20.865990 12508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:20.876796 12508 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38205
I20260806 04:34:20.876873 12702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38205 every 8 connection(s)
I20260806 04:34:20.877751 12508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:20.883582 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12508
I20260806 04:34:20.883677 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:20.883909 12351 external_mini_cluster.cc:1428] Setting key 815eca6cbffa8256732af40729ce348d
Aug 06 04:34:20 dist-test-slave-xt4k krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990860, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:20.891574 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:20.879652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:55253 (local address 127.12.15.254:40883)
0806 04:34:20.879791 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:20.879795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:20.880652 (+ 857us) server_negotiation.cc:407] Connection header received
0806 04:34:20.882028 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:20.882031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:20.882092 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:20.882166 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:20.883371 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:20.883964 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:20.885150 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:20.885380 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:20.885471 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:20.887960 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:20.887964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:20.887967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:20.887995 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:20.889509 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:20.890009 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:20.890014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:20.890015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:20.890120 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:20.890438 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:20.890442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:20.890444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:20.890677 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:20.890779 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:20.891265 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 04:34:20.891401 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:34:20.892410 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:20.880025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40883 (local address 127.12.15.193:55253)
0806 04:34:20.880507 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:20.880551 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:20.881817 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:20.882347 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:20.882354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:20.882723 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:20.883226 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:20.883240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:20.884089 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:20.884092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:20.885026 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:20.885036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:20.885518 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:20.885522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:20.885681 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:20.886211 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:20.886228 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:20.887792 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:20.889629 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:20.889635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:20.889647 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:20.889896 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:20.890209 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:20.890211 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:20.890213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:20.890343 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:20.890782 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:20.890787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:20.891025 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 04:34:20.891181 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 04:34:20.891433 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
I20260806 04:34:20.893743 12703 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40883
I20260806 04:34:20.894003 12703 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:20.894531 12703 heartbeater.cc:511] Master 127.12.15.254:40883 requested a full tablet report, sending...
I20260806 04:34:20.896319 12421 ts_manager.cc:194] Registered new tserver with Master: b5a4aabbac25472da2201de838becc06 (127.12.15.193:38205)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:34:20.897899 12421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:55253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:20.915728 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40883
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:21.038674 12712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:21.038949 12712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:21.039014 12712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:21.043061 12712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:21.043119 12712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:21.043202 12712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:21.048480 12712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:21.049520 12712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:21.050594 12712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:21.060720 12723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:21.060770 12717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:21.060716 12721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:21.061010 12712 server_base.cc:1034] running on GCE node
I20260806 04:34:21.061409 12712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:21.062049 12712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:21.063226 12712 hybrid_clock.cc:648] HybridClock initialized: now 1785990861063200 us; error 47 us; skew 500 ppm
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:21.065939 12712 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:21.066795 12712 webserver.cc:460] Webserver started at http://127.12.15.194:39317/ using document root <none> and password file <none>
I20260806 04:34:21.067338 12712 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:21.067395 12712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:21.067569 12712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:21.069613 12712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "84538e634d2a4c0b8637c7f0a2c82750"
format_stamp: "Formatted at 2026-08-06 04:34:21 on dist-test-slave-xt4k"
server_key: "21c377e7bb766f0b23fa6006187720dd"
server_key_iv: "f1e6e861696ee58acd4d8f63909af9f9"
server_key_version: "encryptionkey@0"
I20260806 04:34:21.070143 12712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "84538e634d2a4c0b8637c7f0a2c82750"
format_stamp: "Formatted at 2026-08-06 04:34:21 on dist-test-slave-xt4k"
server_key: "21c377e7bb766f0b23fa6006187720dd"
server_key_iv: "f1e6e861696ee58acd4d8f63909af9f9"
server_key_version: "encryptionkey@0"
I20260806 04:34:21.073959 12712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:21.077178 12732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:21.079145 12712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:34:21.079290 12712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "84538e634d2a4c0b8637c7f0a2c82750"
format_stamp: "Formatted at 2026-08-06 04:34:21 on dist-test-slave-xt4k"
server_key: "21c377e7bb766f0b23fa6006187720dd"
server_key_iv: "f1e6e861696ee58acd4d8f63909af9f9"
server_key_version: "encryptionkey@0"
I20260806 04:34:21.079414 12712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:21.100752 12712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:21.104661 12712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:21.104854 12712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:21.105540 12712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:21.106384 12712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:21.106431 12712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:21.106485 12712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:21.106514 12712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:21.117308 12712 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:46097
I20260806 04:34:21.117520 12891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:46097 every 8 connection(s)
I20260806 04:34:21.118256 12712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:21.124168 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12712
I20260806 04:34:21.124282 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:21.124598 12351 external_mini_cluster.cc:1428] Setting key 0be95dcd915c452109d04a2c325d0af7
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:21.132907 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:21.120320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:46437 (local address 127.12.15.254:40883)
0806 04:34:21.120482 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:21.120485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:21.121529 (+ 1044us) server_negotiation.cc:407] Connection header received
0806 04:34:21.122887 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:21.122890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:21.122954 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:21.123029 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:21.124212 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.124965 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.126356 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.126663 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.126768 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:21.129474 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:21.129477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:21.129479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:21.129503 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:21.130831 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.131339 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.131342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.131344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.131422 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.131750 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.131753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.131755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.131945 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:21.132032 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:21.132599 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 04:34:21.132742 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 04:34:21.133761 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:21.120746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40883 (local address 127.12.15.194:46437)
0806 04:34:21.121370 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:21.121414 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:21.122672 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:21.123184 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:21.123191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:21.123557 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:21.124059 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:21.124073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.125120 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.125124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:21.126246 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:21.126256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.126828 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.126831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:21.127027 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:21.127615 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:21.127632 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:21.129297 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:21.130949 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:21.130955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:21.130974 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:21.131244 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:21.131506 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:21.131509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:21.131511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:21.131662 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:21.132045 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:21.132052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:21.132314 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 04:34:21.132533 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 04:34:21.132884 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:21.135069 12893 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40883
I20260806 04:34:21.135308 12893 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:21.135859 12893 heartbeater.cc:511] Master 127.12.15.254:40883 requested a full tablet report, sending...
I20260806 04:34:21.137009 12421 ts_manager.cc:194] Registered new tserver with Master: 84538e634d2a4c0b8637c7f0a2c82750 (127.12.15.194:46097)
I20260806 04:34:21.137586 12421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:46437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:21.155189 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40883
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:21.275929 12899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:21.276196 12899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:21.276239 12899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:21.280267 12899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:21.280328 12899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:21.280428 12899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:21.285964 12899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:21.287050 12899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:21.288057 12899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:21.298213 12910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:21.298231 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:21.298221 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:21.298257 12899 server_base.cc:1034] running on GCE node
I20260806 04:34:21.298800 12899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:21.299409 12899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:21.300566 12899 hybrid_clock.cc:648] HybridClock initialized: now 1785990861300549 us; error 32 us; skew 500 ppm
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:21.303215 12899 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:21.304039 12899 webserver.cc:460] Webserver started at http://127.12.15.195:41099/ using document root <none> and password file <none>
I20260806 04:34:21.304605 12899 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:21.304670 12899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:21.304841 12899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:21.306933 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "c7d9ce5e90864c888ae4fdb54fe3f365"
format_stamp: "Formatted at 2026-08-06 04:34:21 on dist-test-slave-xt4k"
server_key: "b0ce9c662774fe9f1b9b39e6dda4e9d5"
server_key_iv: "fc16f2c87b383af2acec833adbed3eef"
server_key_version: "encryptionkey@0"
I20260806 04:34:21.307482 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "c7d9ce5e90864c888ae4fdb54fe3f365"
format_stamp: "Formatted at 2026-08-06 04:34:21 on dist-test-slave-xt4k"
server_key: "b0ce9c662774fe9f1b9b39e6dda4e9d5"
server_key_iv: "fc16f2c87b383af2acec833adbed3eef"
server_key_version: "encryptionkey@0"
I20260806 04:34:21.311121 12899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 04:34:21.313472 12925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:21.314615 12899 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:21.314710 12899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "c7d9ce5e90864c888ae4fdb54fe3f365"
format_stamp: "Formatted at 2026-08-06 04:34:21 on dist-test-slave-xt4k"
server_key: "b0ce9c662774fe9f1b9b39e6dda4e9d5"
server_key_iv: "fc16f2c87b383af2acec833adbed3eef"
server_key_version: "encryptionkey@0"
I20260806 04:34:21.314800 12899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:21.334944 12899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:21.337862 12899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:21.338017 12899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:21.338563 12899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:21.339424 12899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:21.339471 12899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:21.339511 12899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:21.339527 12899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:21.349750 12899 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:38279
I20260806 04:34:21.349788 13088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:38279 every 8 connection(s)
I20260806 04:34:21.350670 12899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:21.352284 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12899
I20260806 04:34:21.352411 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:21.352705 12351 external_mini_cluster.cc:1428] Setting key 9ae4b64c0d5ed4b531b113ccf78ec3ff
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:21.363761 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:21.352540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:45295 (local address 127.12.15.254:40883)
0806 04:34:21.352702 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:21.352707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:21.353547 (+ 840us) server_negotiation.cc:407] Connection header received
0806 04:34:21.354829 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:21.354832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:21.354884 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:21.354959 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:21.356086 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.356721 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.357894 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.358111 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.358198 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:21.360631 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:21.360634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:21.360636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:21.360661 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:21.361826 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.362297 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.362300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.362302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.362380 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.362702 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.362706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.362708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.362885 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:21.362973 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:21.363472 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 04:34:21.363600 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:34:21.364586 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:21.352945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40883 (local address 127.12.15.195:45295)
0806 04:34:21.353400 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:21.353450 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:21.354632 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:21.355141 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:21.355147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:21.355492 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:21.355946 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:21.355959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.356838 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.356841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:21.357779 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:21.357788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.358242 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.358246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:21.358408 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:21.358900 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:21.358916 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:21.360458 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:21.361934 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:21.361940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:21.361954 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:21.362205 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:21.362477 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:21.362480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:21.362482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:21.362615 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:21.362985 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:21.362992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:21.363252 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 04:34:21.363411 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 04:34:21.363718 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":108,"threads_started":1}
I20260806 04:34:21.365664 13089 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40883
I20260806 04:34:21.365903 13089 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:21.366460 13089 heartbeater.cc:511] Master 127.12.15.254:40883 requested a full tablet report, sending...
I20260806 04:34:21.367476 12421 ts_manager.cc:194] Registered new tserver with Master: c7d9ce5e90864c888ae4fdb54fe3f365 (127.12.15.195:38279)
I20260806 04:34:21.368003 12421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:45295
I20260806 04:34:21.378825 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:21.406035 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:21.396988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.12.15.193:38205)
0806 04:34:21.397269 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:21.397278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:21.397302 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:34:21.397354 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:21.397360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:21.397543 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:21.397667 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:21.398266 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.398990 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.399748 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.400102 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.400175 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:21.402694 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:21.402706 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:21.402722 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:21.402746 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:21.404265 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.404699 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.404705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.404706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.404797 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.405057 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.405060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.405062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.405306 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:21.405511 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:21.405724 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 04:34:21.405879 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
W20260806 04:34:21.407050 12575 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51774
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:21.433492 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:21.425644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.12.15.193:38205)
0806 04:34:21.425785 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:21.425790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:21.425908 (+ 118us) server_negotiation.cc:407] Connection header received
0806 04:34:21.426026 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:21.426029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:21.426076 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:21.426145 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:21.426808 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.427333 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.428103 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:21.428380 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:21.428490 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:21.430724 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:21.430727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:21.430729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:21.430756 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:21.432021 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.432411 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.432415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.432417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.432492 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:21.432705 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:21.432707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:21.432708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:21.432866 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:21.432970 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:21.433226 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 04:34:21.433326 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:34:21.434588 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12508
I20260806 04:34:21.447497 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12712
I20260806 04:34:21.456604 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12899
I20260806 04:34:21.476827 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12378
2026-08-06T04:34:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2433 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[13112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[13112](info): set up 2 sockets
Aug 06 04:34:21 dist-test-slave-xt4k krb5kdc[13112](info): commencing operation
krb5kdc: starting...
W20260806 04:34:22.617105 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:22 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:22.745517 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36755
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:22.859611 13133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:22.859885 13133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:22.859926 13133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:22.863968 13133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:22.864040 13133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:22.864059 13133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:22.864074 13133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:22.864089 13133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:22.869357 13133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:22.870365 13133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:22.871332 13133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:22.879828 13144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:22.879905 13146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:22.879906 13143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:22.880203 13133 server_base.cc:1034] running on GCE node
I20260806 04:34:22.880769 13133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:22.881904 13133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:22.883080 13133 hybrid_clock.cc:648] HybridClock initialized: now 1785990862883047 us; error 42 us; skew 500 ppm
Aug 06 04:34:22 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990862, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:22.890182 13133 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:22.891127 13133 webserver.cc:460] Webserver started at http://127.12.15.254:44359/ using document root <none> and password file <none>
I20260806 04:34:22.891692 13133 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:22.891755 13133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:22.891944 13133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:22.894057 13133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "703cf22446da41a187981b8f3252815a"
format_stamp: "Formatted at 2026-08-06 04:34:22 on dist-test-slave-xt4k"
server_key: "eec9e5a7fbb8641cfbffba0459e99ec2"
server_key_iv: "99b78f4be822b62c8cb13561f8f309bd"
server_key_version: "encryptionkey@0"
I20260806 04:34:22.894624 13133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "703cf22446da41a187981b8f3252815a"
format_stamp: "Formatted at 2026-08-06 04:34:22 on dist-test-slave-xt4k"
server_key: "eec9e5a7fbb8641cfbffba0459e99ec2"
server_key_iv: "99b78f4be822b62c8cb13561f8f309bd"
server_key_version: "encryptionkey@0"
I20260806 04:34:22.898293 13133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 04:34:22.900810 13158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:22.902043 13133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:22.902155 13133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "703cf22446da41a187981b8f3252815a"
format_stamp: "Formatted at 2026-08-06 04:34:22 on dist-test-slave-xt4k"
server_key: "eec9e5a7fbb8641cfbffba0459e99ec2"
server_key_iv: "99b78f4be822b62c8cb13561f8f309bd"
server_key_version: "encryptionkey@0"
I20260806 04:34:22.902256 13133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:22.944073 13133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:22.947672 13133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:22.947856 13133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:22.955214 13133 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36755
I20260806 04:34:22.955242 13238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36755 every 8 connection(s)
I20260806 04:34:22.956192 13133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:22.959122 13239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:22.962472 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13133
I20260806 04:34:22.962598 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:22.962823 12351 external_mini_cluster.cc:1428] Setting key c4e3cf8dd1924e36d1d5902e73c3b4e8
I20260806 04:34:22.965472 13239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a: Bootstrap starting.
I20260806 04:34:22.968268 13239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:22.969133 13239 log.cc:824] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:22.971365 13239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a: No bootstrap required, opened a new log
Aug 06 04:34:22 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:22.974766 13239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703cf22446da41a187981b8f3252815a" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36755 } }
I20260806 04:34:22.974993 13239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:22.975028 13239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703cf22446da41a187981b8f3252815a, State: Initialized, Role: FOLLOWER
I20260806 04:34:22.975484 13239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703cf22446da41a187981b8f3252815a" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36755 } }
I20260806 04:34:22.975613 13239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:22.975679 13239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:22.975767 13239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:22.976706 13239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703cf22446da41a187981b8f3252815a" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36755 } }
I20260806 04:34:22.977048 13239 leader_election.cc:302] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 703cf22446da41a187981b8f3252815a; no voters:
I20260806 04:34:22.977356 13239 leader_election.cc:288] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:22.977479 13246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:22.977685 13246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [term 1 LEADER]: Becoming Leader. State: Replica: 703cf22446da41a187981b8f3252815a, State: Running, Role: LEADER
I20260806 04:34:22.978016 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:22.964221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.12.15.254:36755)
0806 04:34:22.964678 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:22.964699 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:22.964742 (+ 43us) server_negotiation.cc:407] Connection header received
0806 04:34:22.965547 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:22.965572 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:22.966011 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:22.966377 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:22.967073 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:22.968564 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:22.969454 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:22.969982 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:22.970151 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:22.972648 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:22.972654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:22.972668 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:22.972695 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:22.975020 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:22.975436 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:22.975442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:22.975447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:22.975570 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:22.975858 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:22.975862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:22.975863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:22.976268 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:22.976495 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:22.976752 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 04:34:22.977074 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":141,"threads_started":1}
I20260806 04:34:22.977999 13246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703cf22446da41a187981b8f3252815a" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36755 } }
I20260806 04:34:22.978700 13239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:22.979235 13246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 703cf22446da41a187981b8f3252815a. Latest consensus state: current_term: 1 leader_uuid: "703cf22446da41a187981b8f3252815a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703cf22446da41a187981b8f3252815a" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36755 } } }
I20260806 04:34:22.979350 13246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:22.980227 13255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:22.981035 13247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "703cf22446da41a187981b8f3252815a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703cf22446da41a187981b8f3252815a" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36755 } } }
I20260806 04:34:22.982227 13247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:22.984591 13255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:22.990707 13255 catalog_manager.cc:1347] Generated new cluster ID: 7bd8f71f573547b7aaa37b7654a68f40
I20260806 04:34:22.990784 13255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:23.000360 13255 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:23.001314 13255 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:23.008283 13255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 703cf22446da41a187981b8f3252815a: Generated new TSK 0
I20260806 04:34:23.009045 13255 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:23.043507 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36755
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:23.162868 13282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:23.163122 13282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:23.163185 13282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:23.167217 13282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:23.167275 13282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:23.167356 13282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:23.172685 13282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36755
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:23.173720 13282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:23.174772 13282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:23.184656 13289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:23.184728 13282 server_base.cc:1034] running on GCE node
W20260806 04:34:23.184682 13288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:23.184656 13292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:23.185259 13282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:23.185953 13282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:23.187134 13282 hybrid_clock.cc:648] HybridClock initialized: now 1785990863187105 us; error 40 us; skew 500 ppm
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:23.189990 13282 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:23.190925 13282 webserver.cc:460] Webserver started at http://127.12.15.193:33231/ using document root <none> and password file <none>
I20260806 04:34:23.191504 13282 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:23.191568 13282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:23.191747 13282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:23.193871 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "f6e10e963c39405491a727c00da1e3dc"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "6a499883e8fe3ea94cfa8165068430bd"
server_key_iv: "a407d309eee349f9075d7dbf715ea10d"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.194411 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "f6e10e963c39405491a727c00da1e3dc"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "6a499883e8fe3ea94cfa8165068430bd"
server_key_iv: "a407d309eee349f9075d7dbf715ea10d"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.198307 13282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:23.201023 13301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.202373 13282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:23.202543 13282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "f6e10e963c39405491a727c00da1e3dc"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "6a499883e8fe3ea94cfa8165068430bd"
server_key_iv: "a407d309eee349f9075d7dbf715ea10d"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.202657 13282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:23.220045 13282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:23.223368 13282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:23.223554 13282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:23.224174 13282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:23.225075 13282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:23.225123 13282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.225168 13282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:23.225198 13282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.234179 13282 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38825
I20260806 04:34:23.234207 13474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38825 every 8 connection(s)
I20260806 04:34:23.235105 13282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:23.241295 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13282
I20260806 04:34:23.241403 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:23.241650 12351 external_mini_cluster.cc:1428] Setting key 4063b2a9c2d4148366d0ab4f2cae1a97
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:23.249214 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.236981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:53001 (local address 127.12.15.254:36755)
0806 04:34:23.237116 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.237119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.237949 (+ 830us) server_negotiation.cc:407] Connection header received
0806 04:34:23.239191 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.239194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.239249 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.239322 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.240508 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.241090 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.242589 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.242820 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.242916 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.245587 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.245591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.245593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.245625 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.247026 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.247592 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.247594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.247595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.247667 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.248058 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.248061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.248062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.248248 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.248328 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.248978 (+ 650us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.249086 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:34:23.250433 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.237321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36755 (local address 127.12.15.193:53001)
0806 04:34:23.237793 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.237837 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.238978 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.239470 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.239477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.239836 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:23.240327 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.240380 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.241240 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.241243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.242457 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.242467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.242968 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.242972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.243141 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.243708 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:23.243806 (+ 98us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:23.245413 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:23.247147 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:23.247153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:23.247169 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:23.247502 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:23.247754 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:23.247757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:23.247760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:23.247948 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:23.248385 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:23.248392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:23.248722 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.248950 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.249324 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":108,"threads_started":1}
I20260806 04:34:23.251748 13477 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36755
I20260806 04:34:23.251981 13477 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:23.252509 13477 heartbeater.cc:511] Master 127.12.15.254:36755 requested a full tablet report, sending...
I20260806 04:34:23.254119 13184 ts_manager.cc:194] Registered new tserver with Master: f6e10e963c39405491a727c00da1e3dc (127.12.15.193:38825)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:34:23.255399 13184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:53001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:23.271975 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36755
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:23.390175 13483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:23.390479 13483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:23.390527 13483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:23.395032 13483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:23.395114 13483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:23.395179 13483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:23.401261 13483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36755
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:23.402424 13483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:23.403477 13483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:34:23.413684 13483 server_base.cc:1034] running on GCE node
W20260806 04:34:23.413595 13490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:23.413595 13489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:23.413599 13494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:23.414278 13483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:23.414930 13483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:23.416105 13483 hybrid_clock.cc:648] HybridClock initialized: now 1785990863416078 us; error 46 us; skew 500 ppm
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:23.419202 13483 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:23.420547 13483 webserver.cc:460] Webserver started at http://127.12.15.194:33225/ using document root <none> and password file <none>
I20260806 04:34:23.421130 13483 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:23.421198 13483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:23.421396 13483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:23.423429 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "af7cff1384d7470bbe23434788297c60"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "158951ac29a353c86a317586697cdf55"
server_key_iv: "230b254f96b19f621888bb3d85944d9b"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.423952 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "af7cff1384d7470bbe23434788297c60"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "158951ac29a353c86a317586697cdf55"
server_key_iv: "230b254f96b19f621888bb3d85944d9b"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.427831 13483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:34:23.430510 13507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.431844 13483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:23.431970 13483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "af7cff1384d7470bbe23434788297c60"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "158951ac29a353c86a317586697cdf55"
server_key_iv: "230b254f96b19f621888bb3d85944d9b"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.432080 13483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:23.448279 13483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:23.451514 13483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:23.451721 13483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:23.452368 13483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:23.453255 13483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:23.453306 13483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.453362 13483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:23.453390 13483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.462507 13483 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:45401
I20260806 04:34:23.462535 13671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:45401 every 8 connection(s)
I20260806 04:34:23.463433 13483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:23.470005 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13483
I20260806 04:34:23.470129 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:23.470389 12351 external_mini_cluster.cc:1428] Setting key 3fa37b86038979e2401b5fac4356f57f
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:23.479032 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.466038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:47493 (local address 127.12.15.254:36755)
0806 04:34:23.466237 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.466240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.467145 (+ 905us) server_negotiation.cc:407] Connection header received
0806 04:34:23.468463 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.468467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.468527 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.468611 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.469881 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.470550 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.471887 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.472120 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.472208 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.474970 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.474973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.474975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.475001 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.476579 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.477320 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.477324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.477325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.477406 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.477763 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.477766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.477768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.477955 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.478067 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.478754 (+ 687us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.478874 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 04:34:23.480501 13676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.466432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36755 (local address 127.12.15.194:47493)
0806 04:34:23.466931 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.466975 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.468207 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.468805 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.468812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.469208 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:23.469727 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.469742 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.470683 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.470686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.471772 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.471783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.472259 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.472263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.472463 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.473053 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:23.473071 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:23.474759 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:23.476886 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:23.476892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:23.476907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:23.477199 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:23.477494 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:23.477497 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:23.477499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:23.477665 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:23.478074 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:23.478081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:23.478436 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.478686 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.479084 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":114,"threads_started":1}
I20260806 04:34:23.481786 13674 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36755
I20260806 04:34:23.482084 13674 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:23.482775 13674 heartbeater.cc:511] Master 127.12.15.254:36755 requested a full tablet report, sending...
I20260806 04:34:23.483979 13184 ts_manager.cc:194] Registered new tserver with Master: af7cff1384d7470bbe23434788297c60 (127.12.15.194:45401)
I20260806 04:34:23.484670 13184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:47493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:23.504588 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36755
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:23.620792 13680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:23.621073 13680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:23.621137 13680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:23.625259 13680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:23.625334 13680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:23.625427 13680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:23.630841 13680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36755
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:23.631953 13680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:23.633029 13680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:23.643215 13687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:23.643221 13688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:23.643249 13691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:23.643472 13680 server_base.cc:1034] running on GCE node
I20260806 04:34:23.644025 13680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:23.644889 13680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:23.646090 13680 hybrid_clock.cc:648] HybridClock initialized: now 1785990863646055 us; error 47 us; skew 500 ppm
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:23.648887 13680 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:23.649757 13680 webserver.cc:460] Webserver started at http://127.12.15.195:38991/ using document root <none> and password file <none>
I20260806 04:34:23.650303 13680 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:23.650372 13680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:23.650574 13680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:23.652925 13680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "fbd0976bbdf54d478fa6020b81421753"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "15d3b478880757c2aa9dcff367a5e4ad"
server_key_iv: "e489345047d90cb6c90105a9a24d5de9"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.653483 13680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "fbd0976bbdf54d478fa6020b81421753"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "15d3b478880757c2aa9dcff367a5e4ad"
server_key_iv: "e489345047d90cb6c90105a9a24d5de9"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.657166 13680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:23.659718 13698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.660964 13680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:23.661096 13680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "fbd0976bbdf54d478fa6020b81421753"
format_stamp: "Formatted at 2026-08-06 04:34:23 on dist-test-slave-xt4k"
server_key: "15d3b478880757c2aa9dcff367a5e4ad"
server_key_iv: "e489345047d90cb6c90105a9a24d5de9"
server_key_version: "encryptionkey@0"
I20260806 04:34:23.661190 13680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:23.678368 13680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:23.681885 13680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:23.682087 13680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:23.682731 13680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:23.683634 13680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:23.683686 13680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.683741 13680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:23.683769 13680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:23.694603 13680 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:44997
I20260806 04:34:23.694643 13864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:44997 every 8 connection(s)
I20260806 04:34:23.695494 13680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:23.702350 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13680
I20260806 04:34:23.702472 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:23.702728 12351 external_mini_cluster.cc:1428] Setting key 3ff99e52a22d7de880b7e5d94d8fce87
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:23.709403 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.697385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:36149 (local address 127.12.15.254:36755)
0806 04:34:23.697522 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.697525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.698380 (+ 855us) server_negotiation.cc:407] Connection header received
0806 04:34:23.699641 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.699644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.699703 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.699777 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.701204 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.701726 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.702960 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.703196 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.703291 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.706098 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.706108 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.706110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.706135 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.707423 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.707925 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.707928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.707929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.708004 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.708314 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.708317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.708319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.708524 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.708612 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.709115 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.709240 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:34:23.710189 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.697739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36755 (local address 127.12.15.195:36149)
0806 04:34:23.698241 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.698282 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.699441 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.699952 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.699958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.700380 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:23.701061 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.701077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.701840 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.701843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.702846 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.702856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.703342 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.703347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.703547 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.704136 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:23.704158 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:23.705948 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:23.707529 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:23.707535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:23.707551 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:23.707833 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:23.708085 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:23.708088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:23.708090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:23.708227 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:23.708642 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:23.708648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:23.708886 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.709052 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.709370 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":157,"threads_started":1}
I20260806 04:34:23.711246 13865 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36755
I20260806 04:34:23.711455 13865 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:23.711912 13865 heartbeater.cc:511] Master 127.12.15.254:36755 requested a full tablet report, sending...
I20260806 04:34:23.712911 13184 ts_manager.cc:194] Registered new tserver with Master: fbd0976bbdf54d478fa6020b81421753 (127.12.15.195:44997)
I20260806 04:34:23.713464 13184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:36149
I20260806 04:34:23.716507 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:23.726734 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.719971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.12.15.254:36755)
0806 04:34:23.720073 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.720077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.720354 (+ 277us) server_negotiation.cc:407] Connection header received
0806 04:34:23.720483 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.720485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.720524 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.720588 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.721295 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.721792 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.722537 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.722736 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.722800 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.724116 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.724119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.724121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.724142 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.725401 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.725799 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.725801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.725802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.725870 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.726059 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.726061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.726062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.726263 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.726360 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.726523 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.726615 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 04:34:23.731518 13184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:23.734560 13184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:23.749366 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.743340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38825 (local address 127.0.0.1:58166)
0806 04:34:23.744137 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.744192 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.744373 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.745457 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.745459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.745476 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:23.745562 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.745568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.746498 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.746501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.747814 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.747825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.748824 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.748828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.748942 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.748981 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.749087 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.749206 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260806 04:34:23.749493 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.744161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.12.15.195:44997)
0806 04:34:23.744765 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.744776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.744796 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:34:23.744868 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.744872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.745017 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.745155 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:23.745437 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.746695 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.747998 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.748904 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.749005 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.749138 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.749254 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.749348 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":174,"threads_started":1}
I20260806 04:34:23.749773 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.743484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.12.15.193:38825)
0806 04:34:23.743889 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.743909 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.744290 (+ 381us) server_negotiation.cc:407] Connection header received
0806 04:34:23.744549 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.744556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.745291 (+ 735us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.745455 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:23.745666 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.746377 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.747953 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.748700 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.748805 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.748964 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.749465 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.749576 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":99,"threads_started":1}
I20260806 04:34:23.749876 13882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.744132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.12.15.194:45401)
0806 04:34:23.744629 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.744640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.744664 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:34:23.744796 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.744801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.745001 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.745154 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:23.745551 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.746684 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.748141 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.748915 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.749004 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.749136 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.749660 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.749753 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":102,"threads_started":1}
I20260806 04:34:23.749558 13881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.743629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:45401 (local address 127.0.0.1:40640)
0806 04:34:23.744555 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.744567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.744666 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.745317 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.745320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.745338 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:23.745428 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.745434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.746836 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.746839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.748029 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.748038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.749198 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.749202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.749291 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.749313 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.749377 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.749442 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":32,"thread_start_us":84,"threads_started":1}
I20260806 04:34:23.749384 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.743415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:44997 (local address 127.0.0.1:38698)
0806 04:34:23.744137 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.744192 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.744326 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.745156 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.745162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.745186 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:23.745327 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.745335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.746898 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.746901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.747902 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.747912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.749057 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.749060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.749142 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.749162 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.749227 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.749290 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":42,"thread_start_us":109,"threads_started":1}
I20260806 04:34:23.753145 13774 tablet_service.cc:1467] Processing CreateTablet for tablet 06acc9b9ba7a4aa4885b66fd9b65b909 (DEFAULT_TABLE table=test-table [id=7b230fa9e87d4da0abc33ad51dbc1e0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:23.753141 13375 tablet_service.cc:1467] Processing CreateTablet for tablet 06acc9b9ba7a4aa4885b66fd9b65b909 (DEFAULT_TABLE table=test-table [id=7b230fa9e87d4da0abc33ad51dbc1e0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:23.754338 13774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06acc9b9ba7a4aa4885b66fd9b65b909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:23.754333 13375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06acc9b9ba7a4aa4885b66fd9b65b909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:23.754053 13576 tablet_service.cc:1467] Processing CreateTablet for tablet 06acc9b9ba7a4aa4885b66fd9b65b909 (DEFAULT_TABLE table=test-table [id=7b230fa9e87d4da0abc33ad51dbc1e0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:23.755087 13576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06acc9b9ba7a4aa4885b66fd9b65b909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:23.761817 13887 tablet_bootstrap.cc:492] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc: Bootstrap starting.
I20260806 04:34:23.762651 13886 tablet_bootstrap.cc:492] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60: Bootstrap starting.
I20260806 04:34:23.762701 13888 tablet_bootstrap.cc:492] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: Bootstrap starting.
I20260806 04:34:23.764274 13887 tablet_bootstrap.cc:654] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:23.764618 13888 tablet_bootstrap.cc:654] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:23.765247 13887 log.cc:824] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:23.765416 13886 tablet_bootstrap.cc:654] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:23.766219 13886 log.cc:824] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:23.766248 13888 log.cc:824] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:23.767398 13887 tablet_bootstrap.cc:492] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc: No bootstrap required, opened a new log
I20260806 04:34:23.767598 13887 ts_tablet_manager.cc:1397] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 04:34:23.768117 13886 tablet_bootstrap.cc:492] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60: No bootstrap required, opened a new log
I20260806 04:34:23.768131 13888 tablet_bootstrap.cc:492] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: No bootstrap required, opened a new log
I20260806 04:34:23.768312 13886 ts_tablet_manager.cc:1397] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:34:23.768312 13888 ts_tablet_manager.cc:1397] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:34:23.770995 13887 raft_consensus.cc:348] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.771214 13887 raft_consensus.cc:374] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:23.771287 13887 raft_consensus.cc:729] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e10e963c39405491a727c00da1e3dc, State: Initialized, Role: FOLLOWER
I20260806 04:34:23.771751 13887 consensus_queue.cc:260] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.772300 13888 raft_consensus.cc:348] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.772652 13888 raft_consensus.cc:374] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:23.772533 13886 raft_consensus.cc:348] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.772745 13888 raft_consensus.cc:729] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd0976bbdf54d478fa6020b81421753, State: Initialized, Role: FOLLOWER
I20260806 04:34:23.772830 13886 raft_consensus.cc:374] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:23.772908 13886 raft_consensus.cc:729] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af7cff1384d7470bbe23434788297c60, State: Initialized, Role: FOLLOWER
I20260806 04:34:23.772976 13477 heartbeater.cc:503] Master 127.12.15.254:36755 was elected leader, sending a full tablet report...
I20260806 04:34:23.773388 13887 ts_tablet_manager.cc:1428] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 04:34:23.773315 13886 consensus_queue.cc:260] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.773315 13888 consensus_queue.cc:260] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.774161 13674 heartbeater.cc:503] Master 127.12.15.254:36755 was elected leader, sending a full tablet report...
I20260806 04:34:23.774289 13888 ts_tablet_manager.cc:1428] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:34:23.774290 13886 ts_tablet_manager.cc:1428] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:34:23.774763 13865 heartbeater.cc:503] Master 127.12.15.254:36755 was elected leader, sending a full tablet report...
I20260806 04:34:23.830658 13899 raft_consensus.cc:482] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:23.830884 13899 raft_consensus.cc:504] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.832033 13899 leader_election.cc:288] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e10e963c39405491a727c00da1e3dc (127.12.15.193:38825), fbd0976bbdf54d478fa6020b81421753 (127.12.15.195:44997)
I20260806 04:34:23.836098 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.832444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:46985 (local address 127.12.15.193:38825)
0806 04:34:23.832603 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.832607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.832786 (+ 179us) server_negotiation.cc:407] Connection header received
0806 04:34:23.832949 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.832952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.833010 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.833110 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:23.833366 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.833967 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.835096 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.835554 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.835646 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.835690 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.835903 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.835977 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:34:23.836090 13676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.832307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38825 (local address 127.12.15.194:46985)
0806 04:34:23.832695 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.832710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.832827 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.833108 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.833111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.833133 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:23.833264 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.833271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.834078 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.834081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.834973 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.834988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.835672 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.835676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.835778 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.835800 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.835872 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.835951 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":33}
I20260806 04:34:23.836195 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.832458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:36295 (local address 127.12.15.195:44997)
0806 04:34:23.832622 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.832627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.833134 (+ 507us) server_negotiation.cc:407] Connection header received
0806 04:34:23.833268 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.833272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.833323 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.833386 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:23.833654 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.834183 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.835248 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.835660 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.835773 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.835820 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.836014 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.836089 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 04:34:23.836367 13901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.832326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:44997 (local address 127.12.15.194:36295)
0806 04:34:23.833039 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:23.833052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:23.833156 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:23.833415 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:23.833417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:23.833434 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:23.833542 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.833548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.834288 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.834291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.835131 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:23.835141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.835795 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.835798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:23.835891 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.835915 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:34:23.835979 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 04:34:23.836044 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":236,"spinlock_wait_cycles":3328,"thread_start_us":153,"threads_started":1}
I20260806 04:34:23.836691 13408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06acc9b9ba7a4aa4885b66fd9b65b909" candidate_uuid: "af7cff1384d7470bbe23434788297c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e10e963c39405491a727c00da1e3dc" is_pre_election: true
I20260806 04:34:23.837023 13408 raft_consensus.cc:2454] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af7cff1384d7470bbe23434788297c60 in term 0.
I20260806 04:34:23.837101 13799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06acc9b9ba7a4aa4885b66fd9b65b909" candidate_uuid: "af7cff1384d7470bbe23434788297c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd0976bbdf54d478fa6020b81421753" is_pre_election: true
I20260806 04:34:23.837378 13799 raft_consensus.cc:2454] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af7cff1384d7470bbe23434788297c60 in term 0.
I20260806 04:34:23.837554 13508 leader_election.cc:302] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af7cff1384d7470bbe23434788297c60, f6e10e963c39405491a727c00da1e3dc; no voters:
I20260806 04:34:23.837847 13899 raft_consensus.cc:2781] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:23.837924 13899 raft_consensus.cc:482] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:23.837952 13899 raft_consensus.cc:3037] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:23.839012 13899 raft_consensus.cc:504] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.839391 13899 leader_election.cc:288] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [CANDIDATE]: Term 1 election: Requested vote from peers f6e10e963c39405491a727c00da1e3dc (127.12.15.193:38825), fbd0976bbdf54d478fa6020b81421753 (127.12.15.195:44997)
I20260806 04:34:23.839864 13799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06acc9b9ba7a4aa4885b66fd9b65b909" candidate_uuid: "af7cff1384d7470bbe23434788297c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd0976bbdf54d478fa6020b81421753"
I20260806 04:34:23.839883 13408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06acc9b9ba7a4aa4885b66fd9b65b909" candidate_uuid: "af7cff1384d7470bbe23434788297c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e10e963c39405491a727c00da1e3dc"
I20260806 04:34:23.839994 13799 raft_consensus.cc:3037] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:23.839994 13408 raft_consensus.cc:3037] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:23.841354 13799 raft_consensus.cc:2454] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af7cff1384d7470bbe23434788297c60 in term 1.
I20260806 04:34:23.841354 13408 raft_consensus.cc:2454] T 06acc9b9ba7a4aa4885b66fd9b65b909 P f6e10e963c39405491a727c00da1e3dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af7cff1384d7470bbe23434788297c60 in term 1.
I20260806 04:34:23.841652 13508 leader_election.cc:302] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af7cff1384d7470bbe23434788297c60, f6e10e963c39405491a727c00da1e3dc; no voters:
I20260806 04:34:23.841859 13899 raft_consensus.cc:2781] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:23.842072 13899 raft_consensus.cc:686] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [term 1 LEADER]: Becoming Leader. State: Replica: af7cff1384d7470bbe23434788297c60, State: Running, Role: LEADER
I20260806 04:34:23.842383 13899 consensus_queue.cc:237] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } }
I20260806 04:34:23.845887 13183 catalog_manager.cc:5495] T 06acc9b9ba7a4aa4885b66fd9b65b909 P af7cff1384d7470bbe23434788297c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to af7cff1384d7470bbe23434788297c60 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "af7cff1384d7470bbe23434788297c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e10e963c39405491a727c00da1e3dc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd0976bbdf54d478fa6020b81421753" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af7cff1384d7470bbe23434788297c60" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45401 } health_report { overall_health: HEALTHY } } }
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:23.874426 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.865553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.12.15.193:38825)
0806 04:34:23.865729 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.865734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.865837 (+ 103us) server_negotiation.cc:407] Connection header received
0806 04:34:23.865978 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.865982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.866027 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.866093 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.866802 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.867300 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.868126 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.868370 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.868464 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.871010 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.871023 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.871041 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.871068 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.872729 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.873137 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.873143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.873144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.873229 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.873480 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.873483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.873485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.873748 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.873956 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.874143 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.874263 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:23.902165 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.894186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.12.15.193:38825)
0806 04:34:23.894357 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.894361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.894452 (+ 91us) server_negotiation.cc:407] Connection header received
0806 04:34:23.894547 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.894550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.894606 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.894671 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.895313 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.895848 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.896735 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.896971 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.897053 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.899271 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.899274 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.899276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.899300 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.900495 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.900972 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.900976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.900977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.901059 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.901307 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.901311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.901312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.901499 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.901620 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.901869 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.901999 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:34:23 dist-test-slave-xt4k krb5kdc[13112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:23.930460 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:23.921009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58200 (local address 127.12.15.193:38825)
0806 04:34:23.921167 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:23.921173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:23.921323 (+ 150us) server_negotiation.cc:407] Connection header received
0806 04:34:23.921470 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:23.921474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:23.921521 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:23.921610 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:23.922314 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.923124 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.924011 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:23.924429 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:23.924570 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:23.927348 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:23.927351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:23.927353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:23.927380 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:23.928747 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.929204 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.929207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.929208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.929291 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:23.929651 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:23.929654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:23.929656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:23.929846 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:23.929956 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:23.930227 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 04:34:23.930333 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 04:34:23.931973 13347 tablet_service.cc:2908] Rejecting scan request for tablet 06acc9b9ba7a4aa4885b66fd9b65b909: Uninitialized: clean time has not yet been initialized
I20260806 04:34:23.934108 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13282
W20260806 04:34:23.946518 13866 tablet.cc:2369] T 06acc9b9ba7a4aa4885b66fd9b65b909 P fbd0976bbdf54d478fa6020b81421753: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:23.950181 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13483
I20260806 04:34:23.960858 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13680
I20260806 04:34:23.992197 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13133
2026-08-06T04:34:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2514 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:24 dist-test-slave-xt4k krb5kdc[13920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:24 dist-test-slave-xt4k krb5kdc[13920](info): set up 2 sockets
Aug 06 04:34:24 dist-test-slave-xt4k krb5kdc[13920](info): commencing operation
krb5kdc: starting...
W20260806 04:34:25.146733 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:25.270709 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40689
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:40689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:25.385470 13938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:25.385766 13938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:25.385824 13938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:25.389860 13938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:25.389921 13938 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 04:34:25.389946 13938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:25.389971 13938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:25.390002 13938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:25.390038 13938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:25.395390 13938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:40689
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:25.396502 13938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:25.397558 13938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:25.406025 13946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:25.406044 13948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:25.406044 13952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:25.406222 13938 server_base.cc:1034] running on GCE node
I20260806 04:34:25.406873 13938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:25.407967 13938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:25.409147 13938 hybrid_clock.cc:648] HybridClock initialized: now 1785990865409125 us; error 45 us; skew 500 ppm
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:25.412113 13938 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:25.413098 13938 webserver.cc:460] Webserver started at http://127.12.15.254:43811/ using document root <none> and password file <none>
I20260806 04:34:25.413653 13938 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:25.413715 13938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:25.413928 13938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:25.416050 13938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "377e55fa1a00425488c5ccf33c4abb7c"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "5f02bee4e6fcf5069cef37cf03a26920"
server_key_iv: "b21831aee22691ea26d83babf6437461"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.416661 13938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "377e55fa1a00425488c5ccf33c4abb7c"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "5f02bee4e6fcf5069cef37cf03a26920"
server_key_iv: "b21831aee22691ea26d83babf6437461"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.420455 13938 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 04:34:25.422868 13963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:25.424084 13938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:25.424206 13938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "377e55fa1a00425488c5ccf33c4abb7c"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "5f02bee4e6fcf5069cef37cf03a26920"
server_key_iv: "b21831aee22691ea26d83babf6437461"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.424304 13938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:25.444128 13938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:25.447126 13938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:25.447311 13938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:25.455473 13938 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:40689
I20260806 04:34:25.455488 14050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:40689 every 8 connection(s)
I20260806 04:34:25.456424 13938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:25.457299 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13938
I20260806 04:34:25.457393 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:25.457635 12351 external_mini_cluster.cc:1428] Setting key 752894ceccd6df2cb6c51de52988430a
I20260806 04:34:25.459518 14051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:25.466146 14051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: Bootstrap starting.
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:25.468729 14051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:25.469904 14051 log.cc:824] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:25.472064 14051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: No bootstrap required, opened a new log
I20260806 04:34:25.472594 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.458984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.12.15.254:40689)
0806 04:34:25.459422 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:25.459442 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:25.459485 (+ 43us) server_negotiation.cc:407] Connection header received
0806 04:34:25.460313 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:25.460366 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:25.460772 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:25.461145 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:25.461787 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.463300 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.464197 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.464911 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.465094 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.467710 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:25.467717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:25.467739 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:25.467773 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:25.469397 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.469927 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.469932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.469937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.470102 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.470431 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.470435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.470437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.470926 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:25.471085 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:25.471445 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 04:34:25.471760 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":111,"threads_started":1}
I20260806 04:34:25.475450 14051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377e55fa1a00425488c5ccf33c4abb7c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40689 } }
I20260806 04:34:25.475672 14051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:25.475734 14051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 377e55fa1a00425488c5ccf33c4abb7c, State: Initialized, Role: FOLLOWER
I20260806 04:34:25.476133 14051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377e55fa1a00425488c5ccf33c4abb7c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40689 } }
I20260806 04:34:25.476259 14051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:25.476318 14051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:25.476438 14051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:25.477281 14051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377e55fa1a00425488c5ccf33c4abb7c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40689 } }
I20260806 04:34:25.477607 14051 leader_election.cc:302] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 377e55fa1a00425488c5ccf33c4abb7c; no voters:
I20260806 04:34:25.477891 14051 leader_election.cc:288] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:25.478017 14057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:25.478555 14057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [term 1 LEADER]: Becoming Leader. State: Replica: 377e55fa1a00425488c5ccf33c4abb7c, State: Running, Role: LEADER
I20260806 04:34:25.478950 14057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377e55fa1a00425488c5ccf33c4abb7c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40689 } }
I20260806 04:34:25.479030 14051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:25.480444 14060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 377e55fa1a00425488c5ccf33c4abb7c. Latest consensus state: current_term: 1 leader_uuid: "377e55fa1a00425488c5ccf33c4abb7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377e55fa1a00425488c5ccf33c4abb7c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40689 } } }
I20260806 04:34:25.480504 14059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "377e55fa1a00425488c5ccf33c4abb7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "377e55fa1a00425488c5ccf33c4abb7c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40689 } } }
I20260806 04:34:25.480583 14060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:25.480583 14059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:25.481024 14081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:25.485455 14081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:25.491534 14081 catalog_manager.cc:1347] Generated new cluster ID: d0e22b5dad924b2990b51764dc09a8c5
I20260806 04:34:25.491612 14081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:25.518507 14081 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:25.519510 14081 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:25.526054 14081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: Generated new TSK 0
I20260806 04:34:25.526733 14081 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:25.563294 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40689
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:25.676977 14093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:25.677260 14093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:25.677306 14093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:25.681476 14093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:25.681540 14093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:25.681581 14093 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:25.681631 14093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:25.687011 14093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:25.688134 14093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:25.689188 14093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:25.698901 14105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:25.699049 14106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:25.698872 14108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:25.699198 14093 server_base.cc:1034] running on GCE node
I20260806 04:34:25.699633 14093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:25.700232 14093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:25.701385 14093 hybrid_clock.cc:648] HybridClock initialized: now 1785990865701368 us; error 35 us; skew 500 ppm
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:25.704198 14093 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:25.705135 14093 webserver.cc:460] Webserver started at http://127.12.15.193:33477/ using document root <none> and password file <none>
I20260806 04:34:25.705694 14093 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:25.705761 14093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:25.705960 14093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:25.707996 14093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "d97571f743bb452483f07bc07a8453b9"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "aa82f99bf301cd2f19d7b340e2f395a3"
server_key_iv: "6b254a8acd9d5f72cd3cebf01d171fec"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.708555 14093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "d97571f743bb452483f07bc07a8453b9"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "aa82f99bf301cd2f19d7b340e2f395a3"
server_key_iv: "6b254a8acd9d5f72cd3cebf01d171fec"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.712210 14093 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:25.714790 14117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:25.716087 14093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:25.716207 14093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "d97571f743bb452483f07bc07a8453b9"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "aa82f99bf301cd2f19d7b340e2f395a3"
server_key_iv: "6b254a8acd9d5f72cd3cebf01d171fec"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.716310 14093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:25.733203 14093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:25.736158 14093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:25.736352 14093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:25.737985 14093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:25.738042 14093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:25.738099 14093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:25.738125 14093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:25.747951 14093 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:45413
I20260806 04:34:25.748111 14280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:45413 every 8 connection(s)
I20260806 04:34:25.749076 14093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:25.750461 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14093
I20260806 04:34:25.750584 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:25.750828 12351 external_mini_cluster.cc:1428] Setting key 80a8d3b1d92be70533fd996ac8d9bf89
I20260806 04:34:25.751299 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.738520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:46177 (local address 127.12.15.254:40689)
0806 04:34:25.738666 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:25.738671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:25.740010 (+ 1339us) server_negotiation.cc:407] Connection header received
0806 04:34:25.741148 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:25.741152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:25.741213 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:25.741310 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:25.742427 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.743241 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.744501 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.744732 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.744824 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.747422 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:25.747425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:25.747427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:25.747452 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:25.749080 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.749688 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.749692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.749694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.749779 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.750134 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.750138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.750140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.750367 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:25.750454 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:25.751000 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 04:34:25.751136 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:34:25.752118 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.739378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.12.15.193:46177)
0806 04:34:25.739849 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:25.739900 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:25.740944 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:25.741448 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:25.741455 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:25.741806 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:25.742282 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.742296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.743376 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.743380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.744384 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.744396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.744886 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.744891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.745059 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.745578 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:25.745594 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:25.747261 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:25.749233 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.749239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.749253 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.749585 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.749863 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.749866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.749868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.750021 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.750465 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:25.750472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:25.750756 (+ 284us) client_negotiation.cc:769] Sending connection context
0806 04:34:25.750918 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 04:34:25.751224 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260806 04:34:25.753242 14281 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40689
I20260806 04:34:25.753383 14281 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:25.753854 14281 heartbeater.cc:511] Master 127.12.15.254:40689 requested a full tablet report, sending...
I20260806 04:34:25.755443 13986 ts_manager.cc:194] Registered new tserver with Master: d97571f743bb452483f07bc07a8453b9 (127.12.15.193:45413)
I20260806 04:34:25.756814 13986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:46177
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:25.762725 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.755571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:48239 (local address 127.12.15.254:40689)
0806 04:34:25.755683 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:25.755688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:25.755903 (+ 215us) server_negotiation.cc:407] Connection header received
0806 04:34:25.756009 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:25.756012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:25.756052 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:25.756133 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:25.756609 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.757296 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.758400 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.758728 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.758840 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.759880 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:25.759885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:25.759888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:25.759913 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:25.761361 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.761790 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.761792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.761793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.761866 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.762073 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.762075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.762076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.762226 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:25.762299 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:25.762525 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 04:34:25.762617 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 04:34:25.762740 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.755486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.12.15.193:48239)
0806 04:34:25.755822 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:25.755835 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:25.755927 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:25.756149 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:25.756152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:25.756370 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:25.756508 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.756515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.757438 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.757442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.758271 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.758282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.758851 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.758854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.758946 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.759433 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:25.759444 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:25.759744 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:25.761467 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.761470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.761473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.761702 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.761936 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.761938 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.761939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.762008 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.762290 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:25.762291 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:25.762370 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 04:34:25.762479 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 04:34:25.762598 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:25.779194 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40689
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:25.890547 14294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:25.890826 14294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:25.890888 14294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:25.895149 14294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:25.895229 14294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:25.895282 14294 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:25.895332 14294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:25.901659 14294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:25.903213 14294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:25.904595 14294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:25.915484 14304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:25.915484 14302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:25.915618 14294 server_base.cc:1034] running on GCE node
W20260806 04:34:25.915789 14301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:25.916199 14294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:25.916930 14294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:25.918102 14294 hybrid_clock.cc:648] HybridClock initialized: now 1785990865918080 us; error 32 us; skew 500 ppm
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:25.920987 14294 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:25.921877 14294 webserver.cc:460] Webserver started at http://127.12.15.194:37451/ using document root <none> and password file <none>
I20260806 04:34:25.922444 14294 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:25.922511 14294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:25.922748 14294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:25.924862 14294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "06b66a2c0d124088a146f78a44a87641"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "b0313db5693df4c11baec8944fc0cf8a"
server_key_iv: "ceb453e82f5956e307f7e8e0004af243"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.925410 14294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "06b66a2c0d124088a146f78a44a87641"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "b0313db5693df4c11baec8944fc0cf8a"
server_key_iv: "ceb453e82f5956e307f7e8e0004af243"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.929220 14294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 04:34:25.931699 14312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:25.933037 14294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:34:25.933164 14294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "06b66a2c0d124088a146f78a44a87641"
format_stamp: "Formatted at 2026-08-06 04:34:25 on dist-test-slave-xt4k"
server_key: "b0313db5693df4c11baec8944fc0cf8a"
server_key_iv: "ceb453e82f5956e307f7e8e0004af243"
server_key_version: "encryptionkey@0"
I20260806 04:34:25.933264 14294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:25.964562 14294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:25.967514 14294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:25.967696 14294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:25.969343 14294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:25.969406 14294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:25.969463 14294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:25.969488 14294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:25 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:25.979691 14294 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:34727
I20260806 04:34:25.979755 14477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:34727 every 8 connection(s)
I20260806 04:34:25.980852 14294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:25.983371 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.970355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:45375 (local address 127.12.15.254:40689)
0806 04:34:25.970506 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:25.970511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:25.971377 (+ 866us) server_negotiation.cc:407] Connection header received
0806 04:34:25.972594 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:25.972598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:25.972664 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:25.972772 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:25.974235 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.974847 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.976122 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.976378 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.976476 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.979647 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:25.979651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:25.979655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:25.979688 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:25.981309 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.981852 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.981855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.981858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.981945 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.982267 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.982270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.982272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.982485 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:25.982584 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:25.983068 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 04:34:25.983204 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:34:25.984146 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.970722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.12.15.194:45375)
0806 04:34:25.971211 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:25.971261 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:25.972355 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:25.972966 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:25.972974 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:25.973364 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:25.973894 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.973909 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.974982 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.974985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.975995 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.976006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.976534 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.976539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.976717 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.977838 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:25.977855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:25.979419 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:25.981446 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.981453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.981468 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.981752 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.982040 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.982042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.982044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.982175 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.982595 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:25.982602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:25.982835 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 04:34:25.983003 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 04:34:25.983341 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":126,"threads_started":1}
I20260806 04:34:25.985322 14478 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40689
I20260806 04:34:25.985461 14478 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:25.986007 14478 heartbeater.cc:511] Master 127.12.15.254:40689 requested a full tablet report, sending...
I20260806 04:34:25.986075 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14294
I20260806 04:34:25.986181 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:25.986873 12351 external_mini_cluster.cc:1428] Setting key 9a1b179f4317deeb3184e2be65eae5a0
I20260806 04:34:25.987564 13987 ts_manager.cc:194] Registered new tserver with Master: 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727)
I20260806 04:34:25.988262 13987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:45375
I20260806 04:34:25.994977 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.987520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.12.15.194:50651)
0806 04:34:25.987890 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:25.987903 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:25.988038 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:25.988305 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:25.988308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:25.988525 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:25.988650 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.988658 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.989594 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.989598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.990352 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:25.990362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.990870 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.990873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:25.990944 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.991419 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:25.991433 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:25.991779 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:25.993515 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.993518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.993520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.993740 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.994019 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:25.994021 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:25.994022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:25.994091 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:25.994560 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:25.994561 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:25.994637 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 04:34:25.994710 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 04:34:25.994820 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":196,"threads_started":1}
I20260806 04:34:25.995081 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:25.987586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:50651 (local address 127.12.15.254:40689)
0806 04:34:25.987693 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:25.987697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:25.987968 (+ 271us) server_negotiation.cc:407] Connection header received
0806 04:34:25.988165 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:25.988169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:25.988215 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:25.988278 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:25.988739 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.989436 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.990449 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:25.990756 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:25.990857 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:25.991913 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:25.991917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:25.991920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:25.991944 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:25.993404 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.993837 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.993840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.993843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.993935 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:25.994165 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:25.994168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:25.994169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:25.994460 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:25.994569 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:25.994790 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 04:34:25.994914 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:26.015833 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40689
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:26.131664 14491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:26.131938 14491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:26.131983 14491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:26.135936 14491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:26.135996 14491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:26.136035 14491 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:26.136072 14491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:26.142026 14491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:26.143131 14491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:26.144173 14491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:26.153813 14499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:26.153813 14498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:26.153910 14502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:26.154273 14491 server_base.cc:1034] running on GCE node
I20260806 04:34:26.154702 14491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:26.155337 14491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:26.156488 14491 hybrid_clock.cc:648] HybridClock initialized: now 1785990866156465 us; error 34 us; skew 500 ppm
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[13920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:26.159253 14491 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:26.160148 14491 webserver.cc:460] Webserver started at http://127.12.15.195:33639/ using document root <none> and password file <none>
I20260806 04:34:26.160730 14491 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:26.160800 14491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:26.161010 14491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:26.163206 14491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "f2be1440539d43b491bc06384d1f8cd8"
format_stamp: "Formatted at 2026-08-06 04:34:26 on dist-test-slave-xt4k"
server_key: "c37973d602b597f9196846786fe78109"
server_key_iv: "92faac632d1231be2730e4a7ddc682b8"
server_key_version: "encryptionkey@0"
I20260806 04:34:26.163741 14491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "f2be1440539d43b491bc06384d1f8cd8"
format_stamp: "Formatted at 2026-08-06 04:34:26 on dist-test-slave-xt4k"
server_key: "c37973d602b597f9196846786fe78109"
server_key_iv: "92faac632d1231be2730e4a7ddc682b8"
server_key_version: "encryptionkey@0"
I20260806 04:34:26.167480 14491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:34:26.170008 14511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:26.171218 14491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:26.171345 14491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "f2be1440539d43b491bc06384d1f8cd8"
format_stamp: "Formatted at 2026-08-06 04:34:26 on dist-test-slave-xt4k"
server_key: "c37973d602b597f9196846786fe78109"
server_key_iv: "92faac632d1231be2730e4a7ddc682b8"
server_key_version: "encryptionkey@0"
I20260806 04:34:26.171444 14491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:26.183475 14491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:26.186703 14491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:26.186884 14491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:26.188397 14491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:26.188450 14491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:26.188505 14491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:26.188535 14491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:26.199138 14491 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:42277
I20260806 04:34:26.199195 14685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:42277 every 8 connection(s)
I20260806 04:34:26.200101 14491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:26.202162 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.189314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:46865 (local address 127.12.15.254:40689)
0806 04:34:26.189472 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.189477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.190346 (+ 869us) server_negotiation.cc:407] Connection header received
0806 04:34:26.191569 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.191573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.191633 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.191736 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:26.193377 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.194084 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.195311 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.195539 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.195628 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.198399 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:26.198402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:26.198404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:26.198430 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:26.200067 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.200707 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.200710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.200712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.200809 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.201197 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.201201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.201202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.201407 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:26.201489 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.201939 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.202040 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:34:26.202881 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.189687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.12.15.195:46865)
0806 04:34:26.190169 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.190227 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.191323 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.191925 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.191935 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.192448 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:26.193188 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.193212 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.194216 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.194220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.195189 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.195204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.195694 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.195699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.195856 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.196469 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:26.196486 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:26.198204 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:26.200195 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:26.200200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:26.200214 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:26.200588 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:26.200896 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:26.200900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:26.200902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:26.201078 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:26.201493 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:26.201498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:26.201718 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.201864 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.202110 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":100,"threads_started":1}
I20260806 04:34:26.203199 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14491
I20260806 04:34:26.203311 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:26.203621 12351 external_mini_cluster.cc:1428] Setting key e95359fc289fbdd333426c5245cdab23
I20260806 04:34:26.204363 14688 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40689
I20260806 04:34:26.204519 14688 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:26.205089 14688 heartbeater.cc:511] Master 127.12.15.254:40689 requested a full tablet report, sending...
I20260806 04:34:26.206187 13986 ts_manager.cc:194] Registered new tserver with Master: f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.206943 13986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:46865
I20260806 04:34:26.207587 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:26.214915 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.206558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:51501 (local address 127.12.15.254:40689)
0806 04:34:26.206667 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.206671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.206895 (+ 224us) server_negotiation.cc:407] Connection header received
0806 04:34:26.206999 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.207002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.207049 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.207118 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:26.209376 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.209902 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.210714 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.210923 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.211008 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.212060 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:26.212064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:26.212066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:26.212090 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:26.213378 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.213769 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.213773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.213774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.213846 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.214089 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.214093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.214094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.214313 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:26.214407 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.214657 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.214761 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:34:26.214926 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.206503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.12.15.195:51501)
0806 04:34:26.206816 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.206828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.206921 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.207142 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.207145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.207476 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:26.209242 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.209251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.210039 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.210042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.210603 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.210612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.211022 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.211026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.211113 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.211594 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:26.211606 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:26.211918 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:26.213478 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:26.213482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:26.213484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:26.213682 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:26.213926 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:26.213929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:26.213931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:26.214009 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:26.214399 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:26.214402 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:26.214495 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.214589 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.214738 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[13920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:26.234403 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.226472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.12.15.254:40689)
0806 04:34:26.226621 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.226624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.226636 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:34:26.226683 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.226685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.226733 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.226796 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:26.227528 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.228055 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.229015 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.229236 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.229311 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.231476 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:26.231480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:26.231482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:26.231506 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:26.232822 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.233207 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.233210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.233211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.233279 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.233550 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.233554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.233555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.233759 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:26.233874 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.234137 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.234238 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:34:26.237493 13987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:26.240628 13987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:26.255631 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.249151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34727 (local address 127.0.0.1:34210)
0806 04:34:26.249658 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.249710 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.249947 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.251051 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.251057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.251084 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.251221 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.251227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.252490 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.252493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.254228 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.254239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.255076 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.255079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.255186 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.255232 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.255340 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.255466 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":110,"thread_start_us":63,"threads_started":1}
I20260806 04:34:26.255636 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.249319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.12.15.194:34727)
0806 04:34:26.249771 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.249783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.250100 (+ 317us) server_negotiation.cc:407] Connection header received
0806 04:34:26.250196 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.250202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.250424 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.250583 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.251346 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.252387 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.254394 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.254959 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.255059 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.255254 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.255393 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.255488 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":118,"threads_started":1}
I20260806 04:34:26.255949 14712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.249318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42277 (local address 127.0.0.1:58980)
0806 04:34:26.250553 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.250577 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.250686 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.251399 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.251402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.251420 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.251517 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.251524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.252774 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.252777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.254224 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.254251 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.255583 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.255587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.255675 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.255697 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.255749 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.255815 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":45,"thread_start_us":117,"threads_started":1}
I20260806 04:34:26.256275 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.249980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:45413 (local address 127.0.0.1:39708)
0806 04:34:26.251001 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.251011 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.251107 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.251641 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.251643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.251661 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.251827 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.251839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.253173 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.253176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.254137 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.254147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.255917 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.255921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.256002 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.256026 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.256085 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.256146 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":60,"spinlock_wait_cycles":1280,"thread_start_us":107,"threads_started":1}
I20260806 04:34:26.256719 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.250534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39708 (local address 127.12.15.193:45413)
0806 04:34:26.250887 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.250900 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.251099 (+ 199us) server_negotiation.cc:407] Connection header received
0806 04:34:26.251230 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.251234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.251396 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.251532 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.251956 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.253045 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.254249 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.255435 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.255583 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.256408 (+ 825us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.256474 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.256596 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":83,"threads_started":1}
I20260806 04:34:26.257758 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.250023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.12.15.195:42277)
0806 04:34:26.250419 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.250430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.250689 (+ 259us) server_negotiation.cc:407] Connection header received
0806 04:34:26.251053 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.251059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.251231 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.251415 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.251622 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.252622 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.254355 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.255435 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.256653 (+ 1218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.257453 (+ 800us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.257512 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.257622 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":91,"threads_started":1}
I20260806 04:34:26.259059 14389 tablet_service.cc:1467] Processing CreateTablet for tablet 90af2ee2e637422f89a84b77590f3409 (DEFAULT_TABLE table=test-table [id=727964200c964098829a06565975bc3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:26.259055 14184 tablet_service.cc:1467] Processing CreateTablet for tablet 90af2ee2e637422f89a84b77590f3409 (DEFAULT_TABLE table=test-table [id=727964200c964098829a06565975bc3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:26.260059 14389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90af2ee2e637422f89a84b77590f3409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.260063 14184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90af2ee2e637422f89a84b77590f3409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.266459 14717 tablet_bootstrap.cc:492] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Bootstrap starting.
I20260806 04:34:26.267287 14585 tablet_service.cc:1467] Processing CreateTablet for tablet 90af2ee2e637422f89a84b77590f3409 (DEFAULT_TABLE table=test-table [id=727964200c964098829a06565975bc3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:26.268453 14585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90af2ee2e637422f89a84b77590f3409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.269126 14717 tablet_bootstrap.cc:654] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.269243 14718 tablet_bootstrap.cc:492] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Bootstrap starting.
I20260806 04:34:26.270449 14717 log.cc:824] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:26.271366 14718 tablet_bootstrap.cc:654] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.272202 14718 log.cc:824] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:26.272681 14717 tablet_bootstrap.cc:492] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: No bootstrap required, opened a new log
I20260806 04:34:26.272874 14717 ts_tablet_manager.cc:1397] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 04:34:26.274230 14718 tablet_bootstrap.cc:492] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: No bootstrap required, opened a new log
I20260806 04:34:26.274416 14718 ts_tablet_manager.cc:1397] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:34:26.275756 14723 tablet_bootstrap.cc:492] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Bootstrap starting.
I20260806 04:34:26.276794 14717 raft_consensus.cc:348] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.277153 14717 raft_consensus.cc:374] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.277240 14717 raft_consensus.cc:729] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97571f743bb452483f07bc07a8453b9, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.277736 14717 consensus_queue.cc:260] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.278245 14723 tablet_bootstrap.cc:654] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.278447 14281 heartbeater.cc:503] Master 127.12.15.254:40689 was elected leader, sending a full tablet report...
I20260806 04:34:26.278771 14717 ts_tablet_manager.cc:1428] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:34:26.278940 14718 raft_consensus.cc:348] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.279201 14718 raft_consensus.cc:374] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.279193 14723 log.cc:824] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:26.279294 14718 raft_consensus.cc:729] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b66a2c0d124088a146f78a44a87641, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.279726 14718 consensus_queue.cc:260] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.280480 14478 heartbeater.cc:503] Master 127.12.15.254:40689 was elected leader, sending a full tablet report...
I20260806 04:34:26.281342 14723 tablet_bootstrap.cc:492] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: No bootstrap required, opened a new log
I20260806 04:34:26.281541 14723 ts_tablet_manager.cc:1397] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:34:26.281719 14718 ts_tablet_manager.cc:1428] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 04:34:26.285218 14723 raft_consensus.cc:348] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.285424 14723 raft_consensus.cc:374] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.285471 14723 raft_consensus.cc:729] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2be1440539d43b491bc06384d1f8cd8, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.285904 14723 consensus_queue.cc:260] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.286628 14688 heartbeater.cc:503] Master 127.12.15.254:40689 was elected leader, sending a full tablet report...
I20260806 04:34:26.286926 14723 ts_tablet_manager.cc:1428] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 04:34:26.390347 14724 raft_consensus.cc:482] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:26.390584 14724 raft_consensus.cc:504] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.391732 14724 leader_election.cc:288] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727), f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.396950 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.392147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:60941 (local address 127.12.15.195:42277)
0806 04:34:26.392273 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.392277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.392560 (+ 283us) server_negotiation.cc:407] Connection header received
0806 04:34:26.392697 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.392700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.392754 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.392841 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.393233 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.394071 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.395701 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.396307 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.396445 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.396499 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.396750 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.396816 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:34:26.396950 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.392101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:56411 (local address 127.12.15.194:34727)
0806 04:34:26.392229 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.392234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.392546 (+ 312us) server_negotiation.cc:407] Connection header received
0806 04:34:26.392691 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.392694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.392752 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.392841 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.393306 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.394071 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.395698 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.396307 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.396444 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.396499 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.396747 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.396816 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:34:26.397032 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.391978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34727 (local address 127.12.15.193:56411)
0806 04:34:26.392413 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.392428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.392582 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.392858 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.392860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.392885 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.393182 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.393190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.394199 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.394203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.395578 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.395583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.396452 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.396456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.396611 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.396647 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.396728 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.396834 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":120,"spinlock_wait_cycles":11264,"thread_start_us":159,"threads_started":1}
I20260806 04:34:26.397142 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.392022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42277 (local address 127.12.15.193:60941)
0806 04:34:26.392449 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.392461 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.392582 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.392858 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.392860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.392885 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.393093 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.393102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.394199 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.394203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.395570 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.395583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.396452 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.396456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.396629 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.396648 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.396740 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.396827 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":134,"spinlock_wait_cycles":30080,"thread_start_us":112,"threads_started":1}
I20260806 04:34:26.397851 14414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90af2ee2e637422f89a84b77590f3409" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b66a2c0d124088a146f78a44a87641" is_pre_election: true
I20260806 04:34:26.397941 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90af2ee2e637422f89a84b77590f3409" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2be1440539d43b491bc06384d1f8cd8" is_pre_election: true
I20260806 04:34:26.398195 14616 raft_consensus.cc:2454] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 0.
I20260806 04:34:26.398196 14414 raft_consensus.cc:2454] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 0.
I20260806 04:34:26.398733 14122 leader_election.cc:302] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b66a2c0d124088a146f78a44a87641, d97571f743bb452483f07bc07a8453b9; no voters:
I20260806 04:34:26.399019 14724 raft_consensus.cc:2781] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:26.399108 14724 raft_consensus.cc:482] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:26.399158 14724 raft_consensus.cc:3037] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.400290 14724 raft_consensus.cc:504] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.400660 14724 leader_election.cc:288] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 election: Requested vote from peers 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727), f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.401000 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90af2ee2e637422f89a84b77590f3409" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2be1440539d43b491bc06384d1f8cd8"
I20260806 04:34:26.400987 14414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90af2ee2e637422f89a84b77590f3409" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b66a2c0d124088a146f78a44a87641"
I20260806 04:34:26.401119 14616 raft_consensus.cc:3037] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.401127 14414 raft_consensus.cc:3037] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.402180 14616 raft_consensus.cc:2454] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 1.
I20260806 04:34:26.402485 14120 leader_election.cc:302] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d97571f743bb452483f07bc07a8453b9, f2be1440539d43b491bc06384d1f8cd8; no voters:
I20260806 04:34:26.402499 14414 raft_consensus.cc:2454] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 1.
I20260806 04:34:26.402654 14724 raft_consensus.cc:2781] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:26.402871 14724 raft_consensus.cc:686] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 1 LEADER]: Becoming Leader. State: Replica: d97571f743bb452483f07bc07a8453b9, State: Running, Role: LEADER
I20260806 04:34:26.403205 14724 consensus_queue.cc:237] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } }
I20260806 04:34:26.406616 13987 catalog_manager.cc:5495] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d97571f743bb452483f07bc07a8453b9 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } health_report { overall_health: HEALTHY } } }
I20260806 04:34:26.448475 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.445119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.12.15.193:45413)
0806 04:34:26.445260 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.445264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.445278 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:26.445425 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.445428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.445480 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.445561 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:26.445805 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.446310 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.447209 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.447418 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.447494 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.447759 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.448130 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.448212 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.448278 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.448329 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 04:34:26.451439 14689 tablet.cc:2369] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:26.454217 14414 raft_consensus.cc:1261] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Refusing update from remote peer d97571f743bb452483f07bc07a8453b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:26.454219 14616 raft_consensus.cc:1261] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Refusing update from remote peer d97571f743bb452483f07bc07a8453b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:26.454838 14732 consensus_queue.cc:1035] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:26.455011 14724 consensus_queue.cc:1035] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:26.462599 14739 mvcc.cc:205] Tried to move back new op lower bound from 7315418588989992960 to 7315418588790353920. Current Snapshot: MvccSnapshot[applied={T|T < 7315418588989992960}]
I20260806 04:34:26.463416 14741 mvcc.cc:205] Tried to move back new op lower bound from 7315418588989992960 to 7315418588790353920. Current Snapshot: MvccSnapshot[applied={T|T < 7315418588989992960}]
I20260806 04:34:26.466063 14740 mvcc.cc:205] Tried to move back new op lower bound from 7315418588989992960 to 7315418588790353920. Current Snapshot: MvccSnapshot[applied={T|T < 7315418588989992960}]
I20260806 04:34:26.478839 13987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:26.479058 13987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:26.481626 13987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: Sending DeleteTablet for 3 replicas of tablet 90af2ee2e637422f89a84b77590f3409
I20260806 04:34:26.482309 14389 tablet_service.cc:1514] Processing DeleteTablet for tablet 90af2ee2e637422f89a84b77590f3409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:26 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:34210
I20260806 04:34:26.483085 13987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 04:34:26.483180 14184 tablet_service.cc:1514] Processing DeleteTablet for tablet 90af2ee2e637422f89a84b77590f3409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:26 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:39708
W20260806 04:34:26.483453 13987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:26.483661 14755 tablet_replica.cc:330] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: stopping tablet replica
I20260806 04:34:26.483800 14585 tablet_service.cc:1514] Processing DeleteTablet for tablet 90af2ee2e637422f89a84b77590f3409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:26 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:58980
I20260806 04:34:26.483954 14755 raft_consensus.cc:2229] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:26.484370 14755 raft_consensus.cc:2258] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:26.484405 14756 tablet_replica.cc:330] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: stopping tablet replica
I20260806 04:34:26.484719 14756 raft_consensus.cc:2229] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:26.485179 14756 raft_consensus.cc:2258] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:26.485993 14755 ts_tablet_manager.cc:1905] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:26.486848 14756 ts_tablet_manager.cc:1905] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:26.488899 14755 ts_tablet_manager.cc:1918] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:26.488952 14585 tablet_service.cc:1467] Processing CreateTablet for tablet 9f443440a0154c14aafabe7c74a97d0f (DEFAULT_TABLE table=test-table [id=b8b54a012cc6434d93e13809812f4e9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:26.489063 14755 log.cc:1196] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/90af2ee2e637422f89a84b77590f3409
I20260806 04:34:26.489231 14585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f443440a0154c14aafabe7c74a97d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.489363 14755 ts_tablet_manager.cc:1939] T 90af2ee2e637422f89a84b77590f3409 P d97571f743bb452483f07bc07a8453b9: Deleting consensus metadata
I20260806 04:34:26.489423 14756 ts_tablet_manager.cc:1918] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:26.489503 14756 log.cc:1196] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/90af2ee2e637422f89a84b77590f3409
I20260806 04:34:26.489776 14756 ts_tablet_manager.cc:1939] T 90af2ee2e637422f89a84b77590f3409 P f2be1440539d43b491bc06384d1f8cd8: Deleting consensus metadata
I20260806 04:34:26.490846 13965 catalog_manager.cc:4841] TS f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277): tablet 90af2ee2e637422f89a84b77590f3409 (table test-table [id=727964200c964098829a06565975bc3c]) successfully deleted
I20260806 04:34:26.491190 14723 tablet_bootstrap.cc:492] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Bootstrap starting.
I20260806 04:34:26.491374 14389 tablet_service.cc:1467] Processing CreateTablet for tablet 9f443440a0154c14aafabe7c74a97d0f (DEFAULT_TABLE table=test-table [id=b8b54a012cc6434d93e13809812f4e9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:26.491591 14389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f443440a0154c14aafabe7c74a97d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.492084 14723 tablet_bootstrap.cc:654] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.493072 14184 tablet_service.cc:1467] Processing CreateTablet for tablet 9f443440a0154c14aafabe7c74a97d0f (DEFAULT_TABLE table=test-table [id=b8b54a012cc6434d93e13809812f4e9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:26.493165 14723 tablet_bootstrap.cc:492] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: No bootstrap required, opened a new log
I20260806 04:34:26.493230 14723 ts_tablet_manager.cc:1397] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 04:34:26.493294 14184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f443440a0154c14aafabe7c74a97d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.493649 14723 raft_consensus.cc:348] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.493747 14723 raft_consensus.cc:374] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.493786 14723 raft_consensus.cc:729] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2be1440539d43b491bc06384d1f8cd8, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.493897 14723 consensus_queue.cc:260] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.494130 14723 ts_tablet_manager.cc:1428] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 04:34:26.495483 14718 tablet_bootstrap.cc:492] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Bootstrap starting.
I20260806 04:34:26.495508 14717 tablet_bootstrap.cc:492] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Bootstrap starting.
I20260806 04:34:26.496404 14718 tablet_bootstrap.cc:654] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.496445 14717 tablet_bootstrap.cc:654] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.496503 14751 tablet_replica.cc:330] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: stopping tablet replica
I20260806 04:34:26.496599 13965 catalog_manager.cc:4841] TS d97571f743bb452483f07bc07a8453b9 (127.12.15.193:45413): tablet 90af2ee2e637422f89a84b77590f3409 (table test-table [id=727964200c964098829a06565975bc3c]) successfully deleted
I20260806 04:34:26.496816 14751 raft_consensus.cc:2229] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:26.497107 14751 raft_consensus.cc:2258] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:26.497750 14718 tablet_bootstrap.cc:492] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: No bootstrap required, opened a new log
I20260806 04:34:26.497834 14718 ts_tablet_manager.cc:1397] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 04:34:26.498271 14718 raft_consensus.cc:348] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.498373 14718 raft_consensus.cc:374] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.498395 14718 raft_consensus.cc:729] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b66a2c0d124088a146f78a44a87641, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.498497 14718 consensus_queue.cc:260] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.498601 14751 ts_tablet_manager.cc:1905] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:26.498759 14718 ts_tablet_manager.cc:1428] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 04:34:26.499575 14717 tablet_bootstrap.cc:492] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: No bootstrap required, opened a new log
I20260806 04:34:26.499657 14717 ts_tablet_manager.cc:1397] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 04:34:26.500126 14717 raft_consensus.cc:348] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.500229 14717 raft_consensus.cc:374] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.500262 14717 raft_consensus.cc:729] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97571f743bb452483f07bc07a8453b9, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.500413 14717 consensus_queue.cc:260] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.500775 14717 ts_tablet_manager.cc:1428] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 04:34:26.500959 14751 ts_tablet_manager.cc:1918] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:26.501040 14751 log.cc:1196] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/90af2ee2e637422f89a84b77590f3409
I20260806 04:34:26.501472 14751 ts_tablet_manager.cc:1939] T 90af2ee2e637422f89a84b77590f3409 P 06b66a2c0d124088a146f78a44a87641: Deleting consensus metadata
I20260806 04:34:26.502604 13967 catalog_manager.cc:4841] TS 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727): tablet 90af2ee2e637422f89a84b77590f3409 (table test-table [id=727964200c964098829a06565975bc3c]) successfully deleted
I20260806 04:34:26.535326 14726 raft_consensus.cc:482] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:26.535480 14726 raft_consensus.cc:504] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.536723 14726 leader_election.cc:288] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97571f743bb452483f07bc07a8453b9 (127.12.15.193:45413), f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.541402 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.537088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:34359 (local address 127.12.15.193:45413)
0806 04:34:26.537208 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.537213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.537441 (+ 228us) server_negotiation.cc:407] Connection header received
0806 04:34:26.537649 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.537653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.537699 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.537744 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.538204 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.538879 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.540228 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.540865 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.540995 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.541051 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.541215 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.541286 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:34:26.541429 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.536943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:45413 (local address 127.12.15.194:34359)
0806 04:34:26.537356 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.537371 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.537516 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.537903 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.537906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.537924 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.538053 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.538059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.538985 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.538989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.540090 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.540100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.540995 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.540999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.541097 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.541121 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.541192 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.541268 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260806 04:34:26.541626 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.537680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:53587 (local address 127.12.15.195:42277)
0806 04:34:26.537792 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.537796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.537809 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:26.537932 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.537935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.537974 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.538033 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:26.538498 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.539271 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.540560 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.540947 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.541066 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.541118 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.541421 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.541501 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:34:26.541980 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.537313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42277 (local address 127.12.15.194:53587)
0806 04:34:26.537733 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.537747 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.537833 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.538063 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.538065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.538081 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:26.538318 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.538327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.539390 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.539394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.540453 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.540462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.541147 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.541151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.541271 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.541296 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.541761 (+ 465us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.541836 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":75,"threads_started":1}
I20260806 04:34:26.542129 14214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f443440a0154c14aafabe7c74a97d0f" candidate_uuid: "06b66a2c0d124088a146f78a44a87641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97571f743bb452483f07bc07a8453b9" is_pre_election: true
I20260806 04:34:26.542415 14214 raft_consensus.cc:2454] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b66a2c0d124088a146f78a44a87641 in term 0.
I20260806 04:34:26.542483 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f443440a0154c14aafabe7c74a97d0f" candidate_uuid: "06b66a2c0d124088a146f78a44a87641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2be1440539d43b491bc06384d1f8cd8" is_pre_election: true
I20260806 04:34:26.542629 14616 raft_consensus.cc:2454] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b66a2c0d124088a146f78a44a87641 in term 0.
I20260806 04:34:26.542853 14315 leader_election.cc:302] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b66a2c0d124088a146f78a44a87641, d97571f743bb452483f07bc07a8453b9; no voters:
I20260806 04:34:26.543087 14726 raft_consensus.cc:2781] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:26.543154 14726 raft_consensus.cc:482] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:26.543191 14726 raft_consensus.cc:3037] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.544677 14726 raft_consensus.cc:504] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.545039 14726 leader_election.cc:288] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [CANDIDATE]: Term 1 election: Requested vote from peers d97571f743bb452483f07bc07a8453b9 (127.12.15.193:45413), f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.545432 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f443440a0154c14aafabe7c74a97d0f" candidate_uuid: "06b66a2c0d124088a146f78a44a87641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2be1440539d43b491bc06384d1f8cd8"
I20260806 04:34:26.545547 14616 raft_consensus.cc:3037] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.546382 14616 raft_consensus.cc:2454] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b66a2c0d124088a146f78a44a87641 in term 1.
I20260806 04:34:26.546638 14315 leader_election.cc:302] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b66a2c0d124088a146f78a44a87641, f2be1440539d43b491bc06384d1f8cd8; no voters:
I20260806 04:34:26.546638 14214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f443440a0154c14aafabe7c74a97d0f" candidate_uuid: "06b66a2c0d124088a146f78a44a87641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97571f743bb452483f07bc07a8453b9"
I20260806 04:34:26.546749 14214 raft_consensus.cc:3037] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.546818 14726 raft_consensus.cc:2781] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:26.546981 14726 raft_consensus.cc:686] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 1 LEADER]: Becoming Leader. State: Replica: 06b66a2c0d124088a146f78a44a87641, State: Running, Role: LEADER
I20260806 04:34:26.547230 14726 consensus_queue.cc:237] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.548559 14214 raft_consensus.cc:2454] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b66a2c0d124088a146f78a44a87641 in term 1.
I20260806 04:34:26.549405 13987 catalog_manager.cc:5495] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06b66a2c0d124088a146f78a44a87641 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "06b66a2c0d124088a146f78a44a87641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } health_report { overall_health: UNKNOWN } } }
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:26.603384 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.594166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.12.15.254:40689)
0806 04:34:26.594311 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.594315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.594505 (+ 190us) server_negotiation.cc:407] Connection header received
0806 04:34:26.594654 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.594656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.594697 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.594763 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:26.595686 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.596209 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.597364 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.597579 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.597649 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.600313 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:26.600316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:26.600318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:26.600365 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:26.601598 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.602132 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.602134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.602136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.602199 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.602482 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.602484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.602485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.602645 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:26.602714 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.603062 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.603251 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":56}
I20260806 04:34:26.603402 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.594072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40689 (local address 127.0.0.1:47840)
0806 04:34:26.594409 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.594427 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.594562 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.594817 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.594821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.595365 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:26.595497 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.595506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.596370 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.596374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.597251 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.597262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.597689 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.597694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.597783 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.598376 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:26.598391 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:26.600175 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:26.601707 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:26.601717 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:26.601724 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:26.602045 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:26.602277 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:26.602280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:26.602282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:26.602386 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:26.602745 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:26.602752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:26.602888 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.603023 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.603196 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":96,"threads_started":1}
I20260806 04:34:26.607601 13987 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47840:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 04:34:26.608414 13987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:26.613346 14389 tablet_service.cc:1467] Processing CreateTablet for tablet 572a0a5b51534c8dbf1d287ee430ae13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=067b739157eb4d0a91101ba05962fa75]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:26.613346 14184 tablet_service.cc:1467] Processing CreateTablet for tablet 572a0a5b51534c8dbf1d287ee430ae13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=067b739157eb4d0a91101ba05962fa75]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:26.613577 14585 tablet_service.cc:1467] Processing CreateTablet for tablet 572a0a5b51534c8dbf1d287ee430ae13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=067b739157eb4d0a91101ba05962fa75]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:26.613781 14184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 572a0a5b51534c8dbf1d287ee430ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.613781 14389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 572a0a5b51534c8dbf1d287ee430ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.613900 14585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 572a0a5b51534c8dbf1d287ee430ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:26.616915 14718 tablet_bootstrap.cc:492] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: Bootstrap starting.
I20260806 04:34:26.617699 14718 tablet_bootstrap.cc:654] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.618206 14717 tablet_bootstrap.cc:492] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: Bootstrap starting.
I20260806 04:34:26.619467 14717 tablet_bootstrap.cc:654] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.620806 14717 tablet_bootstrap.cc:492] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: No bootstrap required, opened a new log
I20260806 04:34:26.620898 14717 ts_tablet_manager.cc:1397] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:26.621223 14718 tablet_bootstrap.cc:492] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: No bootstrap required, opened a new log
I20260806 04:34:26.621292 14718 ts_tablet_manager.cc:1397] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 04:34:26.621423 14717 raft_consensus.cc:348] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.621524 14717 raft_consensus.cc:374] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.621546 14717 raft_consensus.cc:729] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97571f743bb452483f07bc07a8453b9, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.621652 14717 consensus_queue.cc:260] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.621681 14718 raft_consensus.cc:348] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.621778 14718 raft_consensus.cc:374] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.621801 14718 raft_consensus.cc:729] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b66a2c0d124088a146f78a44a87641, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.621906 14718 consensus_queue.cc:260] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.621886 14732 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.622010 14732 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:26.622143 14718 ts_tablet_manager.cc:1428] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:26.622426 14726 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.622552 14726 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:26.622608 14717 ts_tablet_manager.cc:1428] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:26.623526 14723 tablet_bootstrap.cc:492] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: Bootstrap starting.
I20260806 04:34:26.624723 14723 tablet_bootstrap.cc:654] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:26.626245 14723 tablet_bootstrap.cc:492] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: No bootstrap required, opened a new log
I20260806 04:34:26.626339 14723 ts_tablet_manager.cc:1397] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 04:34:26.626820 14723 raft_consensus.cc:348] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.626957 14723 raft_consensus.cc:374] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:26.626989 14723 raft_consensus.cc:729] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2be1440539d43b491bc06384d1f8cd8, State: Initialized, Role: FOLLOWER
I20260806 04:34:26.627115 14723 consensus_queue.cc:260] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.627385 14723 ts_tablet_manager.cc:1428] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:26.627586 14727 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.627740 14727 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:26.699033 14732 raft_consensus.cc:482] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:26.699162 14732 raft_consensus.cc:504] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.699615 14732 leader_election.cc:288] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727), f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.700029 14414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "572a0a5b51534c8dbf1d287ee430ae13" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b66a2c0d124088a146f78a44a87641" is_pre_election: true
I20260806 04:34:26.700070 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "572a0a5b51534c8dbf1d287ee430ae13" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2be1440539d43b491bc06384d1f8cd8" is_pre_election: true
I20260806 04:34:26.700179 14414 raft_consensus.cc:2454] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 0.
I20260806 04:34:26.700197 14616 raft_consensus.cc:2454] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 0.
I20260806 04:34:26.700515 14120 leader_election.cc:302] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d97571f743bb452483f07bc07a8453b9, f2be1440539d43b491bc06384d1f8cd8; no voters:
I20260806 04:34:26.700726 14732 raft_consensus.cc:2781] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:26.700784 14732 raft_consensus.cc:482] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:26.700830 14732 raft_consensus.cc:3037] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.701856 14732 raft_consensus.cc:504] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.702199 14732 leader_election.cc:288] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 election: Requested vote from peers 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727), f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277)
I20260806 04:34:26.702533 14414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "572a0a5b51534c8dbf1d287ee430ae13" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b66a2c0d124088a146f78a44a87641"
I20260806 04:34:26.702561 14616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "572a0a5b51534c8dbf1d287ee430ae13" candidate_uuid: "d97571f743bb452483f07bc07a8453b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2be1440539d43b491bc06384d1f8cd8"
I20260806 04:34:26.702664 14616 raft_consensus.cc:3037] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.702661 14414 raft_consensus.cc:3037] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:26.703723 14414 raft_consensus.cc:2454] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 1.
I20260806 04:34:26.703723 14616 raft_consensus.cc:2454] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97571f743bb452483f07bc07a8453b9 in term 1.
I20260806 04:34:26.704000 14122 leader_election.cc:302] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b66a2c0d124088a146f78a44a87641, d97571f743bb452483f07bc07a8453b9; no voters:
I20260806 04:34:26.704172 14732 raft_consensus.cc:2781] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:26.704272 14732 raft_consensus.cc:686] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [term 1 LEADER]: Becoming Leader. State: Replica: d97571f743bb452483f07bc07a8453b9, State: Running, Role: LEADER
I20260806 04:34:26.704440 14732 consensus_queue.cc:237] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } }
I20260806 04:34:26.705138 14724 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: TxnStatusTablet state changed. Reason: New leader d97571f743bb452483f07bc07a8453b9. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.705245 14724 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:26.705503 14780 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 04:34:26.705672 14780 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 04:34:26.706166 13987 catalog_manager.cc:5495] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d97571f743bb452483f07bc07a8453b9 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:26.727125 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.723677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39724 (local address 127.12.15.193:45413)
0806 04:34:26.723789 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.723793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.723807 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:26.724004 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.724007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.724057 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.724124 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:26.724428 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.724962 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.726148 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.726361 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.726436 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.726735 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.726803 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.726847 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.726949 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.726994 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 04:34:26.727165 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.723561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:45413 (local address 127.0.0.1:39724)
0806 04:34:26.723743 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.723760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.723871 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.724155 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.724158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.724166 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:26.724280 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.724286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.725071 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.725074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.726039 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.726050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.726460 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.726464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.726551 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.726609 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.726866 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.726877 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.726939 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.727025 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 04:34:26.729933 14414 raft_consensus.cc:1261] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Refusing update from remote peer d97571f743bb452483f07bc07a8453b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:26.729954 14616 raft_consensus.cc:1261] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Refusing update from remote peer d97571f743bb452483f07bc07a8453b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:26.730336 14732 consensus_queue.cc:1035] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:26.730515 14724 consensus_queue.cc:1035] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:26.731827 14727 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: TxnStatusTablet state changed. Reason: New leader d97571f743bb452483f07bc07a8453b9. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.731942 14727 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:26.733353 14726 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: TxnStatusTablet state changed. Reason: New leader d97571f743bb452483f07bc07a8453b9. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.733477 14726 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 04:34:26.733680 14479 tablet.cc:2369] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:26.734808 14732 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.734916 14732 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:26.735193 14727 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.735293 14727 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P f2be1440539d43b491bc06384d1f8cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:26.736984 14732 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.737097 14732 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P d97571f743bb452483f07bc07a8453b9: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:26.738230 14726 tablet_replica.cc:439] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d97571f743bb452483f07bc07a8453b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 } } peers { permanent_uuid: "06b66a2c0d124088a146f78a44a87641" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34727 } } peers { permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 } } }
I20260806 04:34:26.738345 14726 tablet_replica.cc:442] T 572a0a5b51534c8dbf1d287ee430ae13 P 06b66a2c0d124088a146f78a44a87641: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:26.744632 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.740773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.12.15.194:34727)
0806 04:34:26.740874 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.740877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.740964 (+ 87us) server_negotiation.cc:407] Connection header received
0806 04:34:26.741105 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.741108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.741167 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.741272 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:26.741448 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.742205 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.743002 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.743334 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.743430 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.743644 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.744183 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.744294 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.744446 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.744501 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 04:34:26.755302 14282 tablet.cc:2369] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:26.755492 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.751542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:51179 (local address 127.12.15.193:45413)
0806 04:34:26.751649 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.751653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.752594 (+ 941us) server_negotiation.cc:407] Connection header received
0806 04:34:26.752707 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.752710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.752756 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.752815 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:26.753071 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.753606 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.754517 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.754740 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.754826 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.755099 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.755166 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.755222 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.755309 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.755359 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 04:34:26.755510 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.751457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:45413 (local address 127.12.15.194:51179)
0806 04:34:26.752512 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.752524 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.752620 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.752831 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.752834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.752860 (+ 26us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:26.752977 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.752982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.753704 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.753708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.754416 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.754428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.754835 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.754838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.754931 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.754980 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.755235 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.755242 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.755299 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.755372 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":103,"threads_started":1}
I20260806 04:34:26.763484 14616 raft_consensus.cc:1261] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Refusing update from remote peer 06b66a2c0d124088a146f78a44a87641: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:26.763616 14214 raft_consensus.cc:1261] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Refusing update from remote peer 06b66a2c0d124088a146f78a44a87641: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:26.763940 14726 consensus_queue.cc:1035] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2be1440539d43b491bc06384d1f8cd8" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:26.764479 14764 consensus_queue.cc:1035] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97571f743bb452483f07bc07a8453b9" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 45413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:26.788381 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.784649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34727 (local address 127.12.15.193:34709)
0806 04:34:26.784962 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:26.784978 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:26.785084 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:26.785290 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:26.785292 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:26.785301 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:26.785407 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.785413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.786187 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.786191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.787070 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:26.787080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.787705 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.787707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:26.787775 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.787816 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.788084 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.788092 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 04:34:26.788150 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 04:34:26.788228 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":104,"threads_started":1}
I20260806 04:34:26.788372 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.784718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:34709 (local address 127.12.15.194:34727)
0806 04:34:26.784849 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.784854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.785048 (+ 194us) server_negotiation.cc:407] Connection header received
0806 04:34:26.785159 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.785161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.785208 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.785273 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:26.785537 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.786094 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.787203 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.787516 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.787594 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.787943 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:26.788011 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:26.788067 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.788159 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.788209 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 04:34:26.820099 13987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:26.820302 13987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:26.822638 13987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 377e55fa1a00425488c5ccf33c4abb7c: Sending DeleteTablet for 3 replicas of tablet 9f443440a0154c14aafabe7c74a97d0f
I20260806 04:34:26.823397 14585 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f443440a0154c14aafabe7c74a97d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:26 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:58980
I20260806 04:34:26.823606 14756 tablet_replica.cc:330] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: stopping tablet replica
I20260806 04:34:26.823724 14756 raft_consensus.cc:2229] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:26.823838 14756 raft_consensus.cc:2258] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:26.824905 14756 ts_tablet_manager.cc:1905] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:26.825758 14389 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f443440a0154c14aafabe7c74a97d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:26 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:34210
I20260806 04:34:26.825773 14184 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f443440a0154c14aafabe7c74a97d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:26 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:39708
I20260806 04:34:26.825927 14751 tablet_replica.cc:330] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: stopping tablet replica
I20260806 04:34:26.826023 14751 raft_consensus.cc:2229] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:26.826315 14751 raft_consensus.cc:2258] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:26.826519 14755 tablet_replica.cc:330] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: stopping tablet replica
I20260806 04:34:26.826656 14755 raft_consensus.cc:2229] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:26.826771 14755 raft_consensus.cc:2258] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:26.828244 14756 ts_tablet_manager.cc:1918] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:26.828318 14756 log.cc:1196] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/9f443440a0154c14aafabe7c74a97d0f
I20260806 04:34:26.828608 14756 ts_tablet_manager.cc:1939] T 9f443440a0154c14aafabe7c74a97d0f P f2be1440539d43b491bc06384d1f8cd8: Deleting consensus metadata
I20260806 04:34:26.829270 13965 catalog_manager.cc:4841] TS f2be1440539d43b491bc06384d1f8cd8 (127.12.15.195:42277): tablet 9f443440a0154c14aafabe7c74a97d0f (table test-table [id=b8b54a012cc6434d93e13809812f4e9d]) successfully deleted
I20260806 04:34:26.829629 14751 ts_tablet_manager.cc:1905] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:26.829629 14755 ts_tablet_manager.cc:1905] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:26.832248 14755 ts_tablet_manager.cc:1918] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:26.832512 14755 log.cc:1196] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/9f443440a0154c14aafabe7c74a97d0f
I20260806 04:34:26.832604 14751 ts_tablet_manager.cc:1918] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:26.832765 14751 log.cc:1196] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/9f443440a0154c14aafabe7c74a97d0f
I20260806 04:34:26.832882 14755 ts_tablet_manager.cc:1939] T 9f443440a0154c14aafabe7c74a97d0f P d97571f743bb452483f07bc07a8453b9: Deleting consensus metadata
I20260806 04:34:26.833069 14751 ts_tablet_manager.cc:1939] T 9f443440a0154c14aafabe7c74a97d0f P 06b66a2c0d124088a146f78a44a87641: Deleting consensus metadata
I20260806 04:34:26.833592 13965 catalog_manager.cc:4841] TS d97571f743bb452483f07bc07a8453b9 (127.12.15.193:45413): tablet 9f443440a0154c14aafabe7c74a97d0f (table test-table [id=b8b54a012cc6434d93e13809812f4e9d]) successfully deleted
I20260806 04:34:26.833709 13967 catalog_manager.cc:4841] TS 06b66a2c0d124088a146f78a44a87641 (127.12.15.194:34727): tablet 9f443440a0154c14aafabe7c74a97d0f (table test-table [id=b8b54a012cc6434d93e13809812f4e9d]) successfully deleted
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[13920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:26.841646 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.828463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39734 (local address 127.12.15.193:45413)
0806 04:34:26.828755 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.828759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.828773 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:26.828811 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.828815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.828858 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.828935 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:26.830850 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.831595 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.833007 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.833321 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.833402 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.837616 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:26.837629 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:26.837646 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:26.837674 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:26.839688 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.840170 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.840177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.840179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.840299 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.840641 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.840644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.840646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.840964 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:26.841172 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.841344 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.841485 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
W20260806 04:34:26.842082 14270 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39734
I20260806 04:34:26.853004 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:26.843843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.12.15.254:40689)
0806 04:34:26.843958 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:26.843962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:26.844395 (+ 433us) server_negotiation.cc:407] Connection header received
0806 04:34:26.844579 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:26.844583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:26.844629 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:26.844677 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:26.845524 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.846333 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.847467 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:26.847823 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:26.847893 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:26.849568 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:26.849572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:26.849575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:26.849603 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:26.851160 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.851651 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.851654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.851656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.851740 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:26.852013 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:26.852016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:26.852017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:26.852222 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:26.852323 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:26.852727 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 04:34:26.852846 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 04:34:26.853580 13987 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47848
I20260806 04:34:26.854883 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14093
I20260806 04:34:26.883798 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14294
I20260806 04:34:26.897723 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14491
I20260806 04:34:26.908443 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13938
2026-08-06T04:34:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2919 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[14807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[14807](info): set up 2 sockets
Aug 06 04:34:26 dist-test-slave-xt4k krb5kdc[14807](info): commencing operation
krb5kdc: starting...
W20260806 04:34:28.060220 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:28.187623 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:45285
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:45285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:28.299199 14824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:28.299464 14824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:28.299531 14824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:28.303530 14824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:28.303586 14824 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 04:34:28.303608 14824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:28.303634 14824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:28.303663 14824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:28.303691 14824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:28.308923 14824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:45285
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:45285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:28.309931 14824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:28.310925 14824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:28.319357 14830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:28.319379 14831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:28.319361 14836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:28.319399 14824 server_base.cc:1034] running on GCE node
I20260806 04:34:28.320150 14824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:28.321234 14824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:28.322384 14824 hybrid_clock.cc:648] HybridClock initialized: now 1785990868322357 us; error 40 us; skew 500 ppm
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:28.328830 14824 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:28.329711 14824 webserver.cc:460] Webserver started at http://127.12.15.254:44693/ using document root <none> and password file <none>
I20260806 04:34:28.330255 14824 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:28.330322 14824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:28.330547 14824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:28.332654 14824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "1a3603ffb7164c9ca750af1573ae0a5c"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "212b49132c9da6237ad1ab29eaf8564e"
server_key_iv: "3813cd2ddb1f134c90a96ddba929e070"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.333212 14824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "1a3603ffb7164c9ca750af1573ae0a5c"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "212b49132c9da6237ad1ab29eaf8564e"
server_key_iv: "3813cd2ddb1f134c90a96ddba929e070"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.336906 14824 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:34:28.339262 14846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:28.340541 14824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:28.340657 14824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "1a3603ffb7164c9ca750af1573ae0a5c"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "212b49132c9da6237ad1ab29eaf8564e"
server_key_iv: "3813cd2ddb1f134c90a96ddba929e070"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.340750 14824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:28.366050 14824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:28.369257 14824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:28.369450 14824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:28.377435 14824 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:45285
I20260806 04:34:28.377480 14936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:45285 every 8 connection(s)
I20260806 04:34:28.378393 14824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:28.381278 14937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:28.384407 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14824
I20260806 04:34:28.384522 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:28.384799 12351 external_mini_cluster.cc:1428] Setting key 0b01633906b78c0950fb8103c0d27c64
I20260806 04:34:28.387455 14937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Bootstrap starting.
I20260806 04:34:28.390038 14937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:28.391026 14937 log.cc:824] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:28.393368 14937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: No bootstrap required, opened a new log
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:28.397571 14937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45285 } }
I20260806 04:34:28.397874 14937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:28.397949 14937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3603ffb7164c9ca750af1573ae0a5c, State: Initialized, Role: FOLLOWER
I20260806 04:34:28.398392 14937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45285 } }
I20260806 04:34:28.398531 14937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:28.398597 14937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:28.398694 14937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:28.399377 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.386164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.12.15.254:45285)
0806 04:34:28.386574 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:28.386590 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:28.386626 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:34:28.387258 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:28.387275 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:28.387642 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:28.387921 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:28.388818 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.389806 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.391000 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.391610 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.391773 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.394343 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:28.394349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:28.394365 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:28.394406 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:28.396222 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.396650 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.396656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.396662 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.396827 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.397153 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.397157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.397159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.397671 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:28.397873 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:28.398373 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 04:34:28.398637 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260806 04:34:28.399559 14937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45285 } }
I20260806 04:34:28.399909 14937 leader_election.cc:302] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a3603ffb7164c9ca750af1573ae0a5c; no voters:
I20260806 04:34:28.400202 14937 leader_election.cc:288] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:28.400300 14943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:28.400525 14943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [term 1 LEADER]: Becoming Leader. State: Replica: 1a3603ffb7164c9ca750af1573ae0a5c, State: Running, Role: LEADER
I20260806 04:34:28.400888 14943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45285 } }
I20260806 04:34:28.401517 14937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:28.402150 14943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a3603ffb7164c9ca750af1573ae0a5c. Latest consensus state: current_term: 1 leader_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45285 } } }
I20260806 04:34:28.402362 14943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:28.402555 14944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3603ffb7164c9ca750af1573ae0a5c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45285 } } }
I20260806 04:34:28.402658 14944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:28.402832 14959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:28.405779 14959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:28.410998 14959 catalog_manager.cc:1347] Generated new cluster ID: 2dc04e1b238e4302a15ce1dcdecf71ba
I20260806 04:34:28.411074 14959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:28.424181 14959 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:28.425489 14959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:28.434757 14959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Generated new TSK 0
I20260806 04:34:28.435469 14959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:28.467744 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:45285
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:28.581452 14977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:28.581773 14977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:28.581842 14977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:28.585909 14977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:28.585983 14977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:28.586041 14977 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 04:34:28.586066 14977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:28.586114 14977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:28.591365 14977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:28.592463 14977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:28.593509 14977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:28.603222 14986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:28.603210 14988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:28.603454 14977 server_base.cc:1034] running on GCE node
W20260806 04:34:28.603210 14983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:28.603960 14977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:28.604609 14977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:28.605764 14977 hybrid_clock.cc:648] HybridClock initialized: now 1785990868605740 us; error 36 us; skew 500 ppm
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:28.608318 14977 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:28.609218 14977 webserver.cc:460] Webserver started at http://127.12.15.193:37129/ using document root <none> and password file <none>
I20260806 04:34:28.609761 14977 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:28.609826 14977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:28.610023 14977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:28.612006 14977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "0818a1c8f3f0431d995da012284def3d"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "fc7d0a22c4847554508209fb28929aee"
server_key_iv: "ed8613e2c75dae7b6f1a8bb15c1842e2"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.612561 14977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "0818a1c8f3f0431d995da012284def3d"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "fc7d0a22c4847554508209fb28929aee"
server_key_iv: "ed8613e2c75dae7b6f1a8bb15c1842e2"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.616058 14977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:34:28.618337 15000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:28.619484 14977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:28.619592 14977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "0818a1c8f3f0431d995da012284def3d"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "fc7d0a22c4847554508209fb28929aee"
server_key_iv: "ed8613e2c75dae7b6f1a8bb15c1842e2"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.619689 14977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:28.643800 14977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:28.646960 14977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:28.647109 14977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:28.648481 14977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:28.648532 14977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:28.648574 14977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:28.648604 14977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:28.658469 14977 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:35601
I20260806 04:34:28.658514 15172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:35601 every 8 connection(s)
I20260806 04:34:28.659456 14977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:28.662206 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.649260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:51095 (local address 127.12.15.254:45285)
0806 04:34:28.649376 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:28.649380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:28.650397 (+ 1017us) server_negotiation.cc:407] Connection header received
0806 04:34:28.651361 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:28.651365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:28.651424 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:28.651479 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:28.653177 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.653991 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.655622 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.655979 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.656101 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.658934 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:28.658938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:28.658940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:28.658964 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:28.660492 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.660924 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.660927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.660928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.661001 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.661251 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.661253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.661254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.661412 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:28.661483 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:28.661977 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 04:34:28.662086 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 04:34:28.662955 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.649780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.12.15.193:51095)
0806 04:34:28.650257 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:28.650298 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:28.651119 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:28.651880 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:28.651889 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:28.652370 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:28.652988 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.653019 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.654144 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.654148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.655490 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.655503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.656153 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.656156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.656303 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.656898 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:28.656920 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:28.658745 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:28.660591 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.660595 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.660604 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.660842 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.661075 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.661076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.661078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.661190 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.661543 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:28.661549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:28.661782 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 04:34:28.661945 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 04:34:28.662181 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":147,"threads_started":1}
I20260806 04:34:28.664126 15173 heartbeater.cc:347] Connected to a master server at 127.12.15.254:45285
I20260806 04:34:28.664258 15173 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:28.664736 15173 heartbeater.cc:511] Master 127.12.15.254:45285 requested a full tablet report, sending...
I20260806 04:34:28.665179 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14977
I20260806 04:34:28.665300 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:28.665606 12351 external_mini_cluster.cc:1428] Setting key d6572008eeae5f7e7aa823d102b8b0c4
I20260806 04:34:28.666215 14875 ts_manager.cc:194] Registered new tserver with Master: 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601)
I20260806 04:34:28.667366 14875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:51095
I20260806 04:34:28.674964 15182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.668056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.12.15.193:53421)
0806 04:34:28.668616 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:28.668630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:28.668724 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:28.668954 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:28.668957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:28.669139 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:28.669232 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.669237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.669956 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.669959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.670502 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.670510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.670904 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.670907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.671000 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.671457 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:28.671473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:28.671880 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:28.673443 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.673447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.673449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.673696 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.673985 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.673987 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.673989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.674066 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.674511 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:28.674514 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:28.674597 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 04:34:28.674691 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 04:34:28.674806 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":85,"threads_started":1}
I20260806 04:34:28.675029 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.668114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:53421 (local address 127.12.15.254:45285)
0806 04:34:28.668460 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:28.668464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:28.668705 (+ 241us) server_negotiation.cc:407] Connection header received
0806 04:34:28.668806 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:28.668808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:28.668846 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:28.668917 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:28.669334 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.669840 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.670603 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.670807 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.670890 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.672016 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:28.672019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:28.672021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:28.672041 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:28.673335 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.673784 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.673787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.673788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.673870 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.674163 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.674167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.674169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.674391 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:28.674497 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:28.674766 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 04:34:28.674877 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:28.695683 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:45285
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:28.808270 15186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:28.808588 15186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:28.808651 15186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:28.812749 15186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:28.812803 15186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:28.812844 15186 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 04:34:28.812870 15186 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:28.812918 15186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:28.818209 15186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:28.819339 15186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:28.820453 15186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:28.830102 15196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:28.830147 15197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:28.830156 15186 server_base.cc:1034] running on GCE node
W20260806 04:34:28.830327 15201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:28.830727 15186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:28.831343 15186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:28.832504 15186 hybrid_clock.cc:648] HybridClock initialized: now 1785990868832485 us; error 27 us; skew 500 ppm
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:28.835196 15186 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:28.836071 15186 webserver.cc:460] Webserver started at http://127.12.15.194:46479/ using document root <none> and password file <none>
I20260806 04:34:28.836683 15186 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:28.836755 15186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:28.836957 15186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:28.838996 15186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "c75e0f9c16534072b93d8a703a5be416"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "4331f0cee35a472a07c73b2609ec1b1f"
server_key_iv: "512eb18c06ccb459d8aae464db8f696e"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.839542 15186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "c75e0f9c16534072b93d8a703a5be416"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "4331f0cee35a472a07c73b2609ec1b1f"
server_key_iv: "512eb18c06ccb459d8aae464db8f696e"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.843326 15186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:34:28.845801 15210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:28.847014 15186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:28.847134 15186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "c75e0f9c16534072b93d8a703a5be416"
format_stamp: "Formatted at 2026-08-06 04:34:28 on dist-test-slave-xt4k"
server_key: "4331f0cee35a472a07c73b2609ec1b1f"
server_key_iv: "512eb18c06ccb459d8aae464db8f696e"
server_key_version: "encryptionkey@0"
I20260806 04:34:28.847239 15186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:28.906090 15186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:28.909268 15186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:28.909458 15186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:28.911005 15186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:28.911059 15186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:28.911118 15186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:28.911144 15186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:28 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:28.921563 15186 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:41877
I20260806 04:34:28.921627 15391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:41877 every 8 connection(s)
I20260806 04:34:28.922480 15186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:28.923007 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 15186
I20260806 04:34:28.923130 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:28.924475 12351 external_mini_cluster.cc:1428] Setting key 691bdae4c9706d002ded110c23c63135
I20260806 04:34:28.925359 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.911849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:56433 (local address 127.12.15.254:45285)
0806 04:34:28.911996 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:28.912001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:28.912963 (+ 962us) server_negotiation.cc:407] Connection header received
0806 04:34:28.914206 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:28.914210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:28.914271 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:28.914369 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:28.915958 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.916761 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.918471 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.918747 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.918832 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.921749 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:28.921754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:28.921756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:28.921786 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:28.923257 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.923840 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.923844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.923846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.923953 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.924286 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.924290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.924292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.924573 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:28.924671 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:28.925120 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 04:34:28.925237 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:34:28.926252 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.912252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.12.15.194:56433)
0806 04:34:28.912791 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:28.912851 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:28.913959 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:28.914586 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:28.914596 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:28.915082 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:28.915809 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.915830 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.916887 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.916891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.918322 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.918335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.918897 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.918902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.919106 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.919762 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:28.919784 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:28.921531 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:28.923359 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.923365 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.923379 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.923726 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.924043 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.924045 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.924047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.924198 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.924669 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:28.924674 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:28.924910 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 04:34:28.925065 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 04:34:28.925368 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":111,"threads_started":1}
I20260806 04:34:28.927420 15392 heartbeater.cc:347] Connected to a master server at 127.12.15.254:45285
I20260806 04:34:28.927683 15392 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:28.928258 15392 heartbeater.cc:511] Master 127.12.15.254:45285 requested a full tablet report, sending...
I20260806 04:34:28.929311 14877 ts_manager.cc:194] Registered new tserver with Master: c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877)
I20260806 04:34:28.930029 14877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:56433
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:28.937966 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.930609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:39387 (local address 127.12.15.254:45285)
0806 04:34:28.930722 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:28.930726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:28.930937 (+ 211us) server_negotiation.cc:407] Connection header received
0806 04:34:28.931050 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:28.931053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:28.931097 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:28.931172 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:28.931655 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.932255 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.933453 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.933767 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.933871 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.935067 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:28.935071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:28.935074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:28.935099 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:28.936390 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.936778 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.936782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.936784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.936861 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:28.937142 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:28.937145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:28.937147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:28.937369 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:28.937464 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:28.937710 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 04:34:28.937813 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:34:28.937973 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:28.930542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.12.15.194:39387)
0806 04:34:28.930833 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:28.930848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:28.930954 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:28.931205 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:28.931208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:28.931424 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:28.931541 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.931549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.932413 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.932417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.933227 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:28.933239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:28.933985 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:28.933988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:28.934064 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:28.934549 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:28.934564 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:28.934906 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:28.936483 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.936487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.936489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.936690 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.936940 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:28.936944 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:28.936946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:28.937057 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:28.937453 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:28.937457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:28.937556 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 04:34:28.937641 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:34:28.937783 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:28.955883 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:45285
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:29.069093 15403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:29.069348 15403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:29.069420 15403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:29.073328 15403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:29.073387 15403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:29.073451 15403 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 04:34:29.073477 15403 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:29.073521 15403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:29.078768 15403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:45285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:29.079800 15403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:29.080832 15403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:29.090384 15417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:29.090415 15410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:29.090476 15403 server_base.cc:1034] running on GCE node
W20260806 04:34:29.090421 15413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:29.091025 15403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:29.091625 15403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:29.092782 15403 hybrid_clock.cc:648] HybridClock initialized: now 1785990869092760 us; error 31 us; skew 500 ppm
Aug 06 04:34:29 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990869, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:29.095438 15403 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:29.096297 15403 webserver.cc:460] Webserver started at http://127.12.15.195:38597/ using document root <none> and password file <none>
I20260806 04:34:29.096858 15403 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:29.096920 15403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:29.097134 15403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:29.099143 15403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "cc971183b2034bfe9b700c80b84606fd"
format_stamp: "Formatted at 2026-08-06 04:34:29 on dist-test-slave-xt4k"
server_key: "9017ab1306f64fd29d04057c3a858a42"
server_key_iv: "9d8e236cb330ac07b12a396f1a6ef925"
server_key_version: "encryptionkey@0"
I20260806 04:34:29.099673 15403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "cc971183b2034bfe9b700c80b84606fd"
format_stamp: "Formatted at 2026-08-06 04:34:29 on dist-test-slave-xt4k"
server_key: "9017ab1306f64fd29d04057c3a858a42"
server_key_iv: "9d8e236cb330ac07b12a396f1a6ef925"
server_key_version: "encryptionkey@0"
I20260806 04:34:29.103526 15403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:34:29.105880 15427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:29.107071 15403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:29.107194 15403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "cc971183b2034bfe9b700c80b84606fd"
format_stamp: "Formatted at 2026-08-06 04:34:29 on dist-test-slave-xt4k"
server_key: "9017ab1306f64fd29d04057c3a858a42"
server_key_iv: "9d8e236cb330ac07b12a396f1a6ef925"
server_key_version: "encryptionkey@0"
I20260806 04:34:29.107288 15403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:29.118448 15403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:29.121481 15403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:29.121655 15403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:29.123137 15403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:29.123199 15403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:29.123253 15403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:29.123281 15403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:29 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990869, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:29.133474 15403 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:41009
I20260806 04:34:29.133510 15595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:41009 every 8 connection(s)
I20260806 04:34:29.134649 15403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:29.136956 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.123930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:37929 (local address 127.12.15.254:45285)
0806 04:34:29.124088 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.124094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.125010 (+ 916us) server_negotiation.cc:407] Connection header received
0806 04:34:29.126049 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.126054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.126118 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.126223 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:29.127632 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.128240 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.129960 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.130283 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.130400 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.133329 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:29.133333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:29.133336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:29.133369 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:29.134908 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.135498 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.135502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.135504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.135628 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.135947 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.135949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.135951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.136126 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:29.136200 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.136728 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.136833 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:34:29.137737 15437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.124392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.12.15.195:37929)
0806 04:34:29.124851 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.124900 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.125780 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.126399 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.126409 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.126867 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:29.127460 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.127478 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.128382 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.128386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.129830 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.129842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.130441 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.130446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.130674 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.131261 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:29.131281 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:29.133116 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:29.135028 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:29.135034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:29.135051 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:29.135377 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:29.135726 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:29.135730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:29.135732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:29.135869 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:29.136202 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:29.136207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:29.136486 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.136640 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.136887 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":110,"threads_started":1}
I20260806 04:34:29.138833 15596 heartbeater.cc:347] Connected to a master server at 127.12.15.254:45285
I20260806 04:34:29.139070 15596 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:29.139580 15596 heartbeater.cc:511] Master 127.12.15.254:45285 requested a full tablet report, sending...
I20260806 04:34:29.140666 14875 ts_manager.cc:194] Registered new tserver with Master: cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.141227 14875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:37929
I20260806 04:34:29.142195 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 15403
I20260806 04:34:29.142325 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:29.142712 12351 external_mini_cluster.cc:1428] Setting key ba3d81392cdc65f8b72e2f5610afa068
I20260806 04:34:29.144754 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:29.149613 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.141706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.12.15.195:45557)
0806 04:34:29.141989 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.142004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.142107 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.142460 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.142463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.142678 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:29.142803 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.142810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.143727 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.143730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.144451 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.144462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.145044 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.145048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.145141 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.145661 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:29.145674 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:29.146093 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:29.147923 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:29.147926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:29.147929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:29.148177 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:29.148518 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:29.148521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:29.148523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:29.148621 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:29.149057 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:29.149060 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:29.149160 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.149259 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.149399 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260806 04:34:29.149624 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.141807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:45557 (local address 127.12.15.254:45285)
0806 04:34:29.141921 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.141947 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.142075 (+ 128us) server_negotiation.cc:407] Connection header received
0806 04:34:29.142193 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.142196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.142243 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.142322 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:29.142941 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.143578 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.144604 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.144891 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.144994 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.146251 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:29.146254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:29.146256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:29.146278 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:29.147794 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.148282 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.148286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.148288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.148423 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.148716 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.148720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.148721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.148952 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:29.149074 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.149333 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.149457 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 04:34:29 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:34:29 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:29.173377 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.165549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.12.15.254:45285)
0806 04:34:29.165682 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.165685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.165700 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:34:29.165745 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.165748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.165788 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.165859 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:29.166575 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.167110 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.167978 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.168180 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.168248 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.170575 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:29.170580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:29.170583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:29.170610 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:29.171885 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.172296 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.172299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.172301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.172425 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.172664 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.172666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.172668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.172848 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:29.172949 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.173114 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.173214 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:34:29.176265 14875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:29.179235 14875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:29.194702 15621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.187845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35601 (local address 127.0.0.1:55946)
0806 04:34:29.188997 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.189008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.189079 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.189867 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.189869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.189887 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.190035 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.190041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.191351 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.191355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.192774 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.192786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.194095 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.194099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.194225 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.194275 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.194398 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.194532 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260806 04:34:29.194896 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.187622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41009 (local address 127.0.0.1:45718)
0806 04:34:29.188590 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.188638 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.188766 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.189329 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.189336 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.189383 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.189592 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.189602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.191526 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.191530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.193249 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.193283 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.194498 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.194501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.194589 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.194612 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.194690 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.194759 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260806 04:34:29.195293 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.188398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.12.15.195:41009)
0806 04:34:29.188798 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.188809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.188833 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:34:29.188907 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.188912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.189136 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.189295 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.189807 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.191343 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.193403 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.194171 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.194260 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.194435 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.195059 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.195160 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":96,"threads_started":1}
I20260806 04:34:29.195320 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.188388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55946 (local address 127.12.15.193:35601)
0806 04:34:29.189295 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.189302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.189321 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:34:29.189388 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.189392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.189578 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.189714 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.190167 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.191213 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.192937 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.193745 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.193856 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.194014 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.195090 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.195193 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":103,"threads_started":1}
I20260806 04:34:29.196300 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.187578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.0.0.1:42946)
0806 04:34:29.188683 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.188698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.188785 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.189793 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.189796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.189813 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.190109 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.190120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.192241 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.192245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.193500 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.193509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.195906 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.195910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.196002 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.196028 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.196104 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.196171 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":322,"thread_start_us":84,"threads_started":1}
I20260806 04:34:29.196368 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.187750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.12.15.194:41877)
0806 04:34:29.188992 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.189000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.189019 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:34:29.189076 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.189080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.189215 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.189644 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.190250 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.192115 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.193624 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.195738 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.195841 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.195990 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.196157 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.196235 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1139,"thread_start_us":88,"threads_started":1}
I20260806 04:34:29.198735 15071 tablet_service.cc:1467] Processing CreateTablet for tablet 658cc2e71d2a4e72b860511c90cc1d1e (DEFAULT_TABLE table=test-table [id=4621eb1d170d4bf1b43d3c2ab5ec3ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:29.199820 15071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658cc2e71d2a4e72b860511c90cc1d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.199803 15292 tablet_service.cc:1467] Processing CreateTablet for tablet 658cc2e71d2a4e72b860511c90cc1d1e (DEFAULT_TABLE table=test-table [id=4621eb1d170d4bf1b43d3c2ab5ec3ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:29.200964 15292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658cc2e71d2a4e72b860511c90cc1d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.202889 15491 tablet_service.cc:1467] Processing CreateTablet for tablet 658cc2e71d2a4e72b860511c90cc1d1e (DEFAULT_TABLE table=test-table [id=4621eb1d170d4bf1b43d3c2ab5ec3ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:29.203924 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 658cc2e71d2a4e72b860511c90cc1d1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.206681 15630 tablet_bootstrap.cc:492] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Bootstrap starting.
I20260806 04:34:29.208846 15631 tablet_bootstrap.cc:492] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Bootstrap starting.
I20260806 04:34:29.208884 15630 tablet_bootstrap.cc:654] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.210177 15630 log.cc:824] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:29.211175 15631 tablet_bootstrap.cc:654] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.211750 15634 tablet_bootstrap.cc:492] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Bootstrap starting.
I20260806 04:34:29.212049 15631 log.cc:824] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:29.212466 15630 tablet_bootstrap.cc:492] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: No bootstrap required, opened a new log
I20260806 04:34:29.212661 15630 ts_tablet_manager.cc:1397] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:34:29.213611 15634 tablet_bootstrap.cc:654] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.214126 15631 tablet_bootstrap.cc:492] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: No bootstrap required, opened a new log
I20260806 04:34:29.214323 15631 ts_tablet_manager.cc:1397] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 04:34:29.214304 15634 log.cc:824] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:29.216285 15634 tablet_bootstrap.cc:492] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: No bootstrap required, opened a new log
I20260806 04:34:29.216492 15634 ts_tablet_manager.cc:1397] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:34:29.217255 15630 raft_consensus.cc:348] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.217546 15630 raft_consensus.cc:374] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.217643 15630 raft_consensus.cc:729] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.218115 15630 consensus_queue.cc:260] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.218115 15631 raft_consensus.cc:348] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.218405 15631 raft_consensus.cc:374] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.218472 15631 raft_consensus.cc:729] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.218873 15631 consensus_queue.cc:260] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.219112 15630 ts_tablet_manager.cc:1428] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:34:29.219497 15173 heartbeater.cc:503] Master 127.12.15.254:45285 was elected leader, sending a full tablet report...
I20260806 04:34:29.219583 15634 raft_consensus.cc:348] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.219753 15631 ts_tablet_manager.cc:1428] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260806 04:34:29.219816 15634 raft_consensus.cc:374] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.219916 15634 raft_consensus.cc:729] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc971183b2034bfe9b700c80b84606fd, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.220070 15392 heartbeater.cc:503] Master 127.12.15.254:45285 was elected leader, sending a full tablet report...
I20260806 04:34:29.220328 15634 consensus_queue.cc:260] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.220945 15596 heartbeater.cc:503] Master 127.12.15.254:45285 was elected leader, sending a full tablet report...
I20260806 04:34:29.222100 15634 ts_tablet_manager.cc:1428] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:34:29.241672 15638 raft_consensus.cc:482] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:29.241920 15638 raft_consensus.cc:504] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.243120 15638 leader_election.cc:288] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.247288 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.243476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:54661 (local address 127.12.15.194:41877)
0806 04:34:29.243612 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.243617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.243964 (+ 347us) server_negotiation.cc:407] Connection header received
0806 04:34:29.244070 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.244073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.244123 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.244207 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.244514 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.245201 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.246311 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.246709 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.246906 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.246958 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.247081 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.247157 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:34:29.247293 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.243559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:59493 (local address 127.12.15.195:41009)
0806 04:34:29.243701 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.243808 (+ 107us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.244062 (+ 254us) server_negotiation.cc:407] Connection header received
0806 04:34:29.244190 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.244193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.244243 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.244317 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.244624 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.245221 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.246302 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.246699 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.246791 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.246845 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.247115 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.247190 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 04:34:29.247318 15645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.243355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.12.15.193:54661)
0806 04:34:29.243859 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.243876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.243987 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.244227 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.244230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.244247 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.244403 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.244411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.245305 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.245308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.246187 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.246197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.246874 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.246878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.246959 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.246994 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.247047 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.247122 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":44,"thread_start_us":131,"threads_started":1}
I20260806 04:34:29.247325 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.243390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41009 (local address 127.12.15.193:59493)
0806 04:34:29.243956 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.243972 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.244081 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.244367 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.244370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.244387 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.244503 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.244509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.245340 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.245343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.246181 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.246191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.246850 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.246854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.246954 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.247010 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.247100 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.247160 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":23,"thread_start_us":117,"threads_started":1}
I20260806 04:34:29.248010 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658cc2e71d2a4e72b860511c90cc1d1e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75e0f9c16534072b93d8a703a5be416" is_pre_election: true
I20260806 04:34:29.248278 15522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658cc2e71d2a4e72b860511c90cc1d1e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd" is_pre_election: true
I20260806 04:34:29.248322 15325 raft_consensus.cc:2454] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 0.
I20260806 04:34:29.248579 15522 raft_consensus.cc:2454] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 0.
I20260806 04:34:29.248800 15002 leader_election.cc:302] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0818a1c8f3f0431d995da012284def3d, c75e0f9c16534072b93d8a703a5be416; no voters:
I20260806 04:34:29.249111 15638 raft_consensus.cc:2781] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:29.249190 15638 raft_consensus.cc:482] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:29.249241 15638 raft_consensus.cc:3037] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.250236 15638 raft_consensus.cc:504] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.250558 15638 leader_election.cc:288] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 election: Requested vote from peers c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.250895 15522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658cc2e71d2a4e72b860511c90cc1d1e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd"
I20260806 04:34:29.250970 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "658cc2e71d2a4e72b860511c90cc1d1e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75e0f9c16534072b93d8a703a5be416"
I20260806 04:34:29.251029 15522 raft_consensus.cc:3037] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.251080 15325 raft_consensus.cc:3037] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.252113 15325 raft_consensus.cc:2454] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 1.
I20260806 04:34:29.252120 15522 raft_consensus.cc:2454] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 1.
I20260806 04:34:29.252418 15001 leader_election.cc:302] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0818a1c8f3f0431d995da012284def3d, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:29.252596 15638 raft_consensus.cc:2781] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:29.252871 15638 raft_consensus.cc:686] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 1 LEADER]: Becoming Leader. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Running, Role: LEADER
I20260806 04:34:29.253194 15638 consensus_queue.cc:237] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } }
I20260806 04:34:29.256305 14865 catalog_manager.cc:5495] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0818a1c8f3f0431d995da012284def3d (127.12.15.193). New cstate: current_term: 1 leader_uuid: "0818a1c8f3f0431d995da012284def3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } health_report { overall_health: HEALTHY } } }
Aug 06 04:34:29 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990868, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:29.297785 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.290023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:45285 (local address 127.0.0.1:38456)
0806 04:34:29.290320 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.290332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.290440 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.290678 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.290680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.291095 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:29.291195 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.291201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.291917 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.291920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.292516 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.292526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.292923 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.292926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.292999 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.293541 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:29.293553 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:29.294976 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:29.296459 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:29.296467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:29.296473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:29.296708 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:29.296922 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:29.296924 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:29.296925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:29.296995 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:29.297286 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:29.297291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:29.297395 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.297480 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.297619 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":122,"threads_started":1}
I20260806 04:34:29.297798 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.290095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.12.15.254:45285)
0806 04:34:29.290238 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.290243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.290398 (+ 155us) server_negotiation.cc:407] Connection header received
0806 04:34:29.290535 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.290545 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.290584 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.290646 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:29.291303 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.291820 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.292624 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.292827 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.292913 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.295114 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:29.295118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:29.295120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:29.295142 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:29.296353 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.296786 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.296788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.296789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.296859 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:29.297055 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:29.297057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:29.297058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:29.297216 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:29.297285 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.297521 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.297673 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":57}
I20260806 04:34:29.301755 14865 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:38456:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 04:34:29.302480 14865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:29.307288 15071 tablet_service.cc:1467] Processing CreateTablet for tablet 20a23a7c414e44b992ea434ee3b1f9db (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20d10ae602c640acb6ae5c5b3db30be9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:29.307497 15491 tablet_service.cc:1467] Processing CreateTablet for tablet 20a23a7c414e44b992ea434ee3b1f9db (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20d10ae602c640acb6ae5c5b3db30be9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:29.307488 15292 tablet_service.cc:1467] Processing CreateTablet for tablet 20a23a7c414e44b992ea434ee3b1f9db (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20d10ae602c640acb6ae5c5b3db30be9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:29.308058 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a23a7c414e44b992ea434ee3b1f9db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.308068 15071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a23a7c414e44b992ea434ee3b1f9db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.308138 15292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a23a7c414e44b992ea434ee3b1f9db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.310587 15631 tablet_bootstrap.cc:492] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: Bootstrap starting.
I20260806 04:34:29.310973 15630 tablet_bootstrap.cc:492] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: Bootstrap starting.
I20260806 04:34:29.311527 15631 tablet_bootstrap.cc:654] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.312180 15630 tablet_bootstrap.cc:654] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.312659 15634 tablet_bootstrap.cc:492] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: Bootstrap starting.
I20260806 04:34:29.313040 15631 tablet_bootstrap.cc:492] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: No bootstrap required, opened a new log
I20260806 04:34:29.313102 15631 ts_tablet_manager.cc:1397] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 04:34:29.313462 15634 tablet_bootstrap.cc:654] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.313446 15631 raft_consensus.cc:348] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.313524 15631 raft_consensus.cc:374] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.313544 15631 raft_consensus.cc:729] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.313649 15631 consensus_queue.cc:260] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.313844 15631 ts_tablet_manager.cc:1428] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 04:34:29.313889 15639 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.314054 15639 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.314798 15634 tablet_bootstrap.cc:492] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: No bootstrap required, opened a new log
I20260806 04:34:29.314879 15634 ts_tablet_manager.cc:1397] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:29.315234 15634 raft_consensus.cc:348] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.315321 15634 raft_consensus.cc:374] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.315349 15634 raft_consensus.cc:729] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc971183b2034bfe9b700c80b84606fd, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.315470 15634 consensus_queue.cc:260] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.315692 15634 ts_tablet_manager.cc:1428] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:29.315949 15643 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.316093 15643 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.316673 15630 tablet_bootstrap.cc:492] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: No bootstrap required, opened a new log
I20260806 04:34:29.316759 15630 ts_tablet_manager.cc:1397] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 04:34:29.317221 15630 raft_consensus.cc:348] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.317307 15630 raft_consensus.cc:374] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.317328 15630 raft_consensus.cc:729] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.317425 15630 consensus_queue.cc:260] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.317651 15638 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.317791 15638 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.317665 15630 ts_tablet_manager.cc:1428] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260806 04:34:29.385740 15597 tablet.cc:2369] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:34:29.411377 15174 tablet.cc:2369] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:34:29.424314 15393 tablet.cc:2369] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:29.453413 15639 raft_consensus.cc:482] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:29.453577 15639 raft_consensus.cc:504] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.454878 15639 leader_election.cc:288] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.459872 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.455252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:53995 (local address 127.12.15.193:35601)
0806 04:34:29.455386 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.455391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.455786 (+ 395us) server_negotiation.cc:407] Connection header received
0806 04:34:29.455869 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.455872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.455917 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.455974 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.456182 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.456914 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.458552 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.459161 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.459283 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.459337 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.459668 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.459737 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:34:29.459872 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.455277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:43843 (local address 127.12.15.195:41009)
0806 04:34:29.455403 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.455408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.455492 (+ 84us) server_negotiation.cc:407] Connection header received
0806 04:34:29.455627 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.455630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.455686 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.455763 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:29.456043 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.456832 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.458552 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.459160 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.459283 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.459337 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.459669 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.459735 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:34:29.459934 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.455153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41009 (local address 127.12.15.194:43843)
0806 04:34:29.455401 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.455413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.455526 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.455791 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.455794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.455808 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.455926 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.455939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.456976 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.456980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.458415 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.458427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.459307 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.459312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.459518 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.459570 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.459655 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.459746 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":125,"thread_start_us":72,"threads_started":1}
I20260806 04:34:29.459935 15675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.455153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35601 (local address 127.12.15.194:53995)
0806 04:34:29.455707 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.455717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.455790 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.456000 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.456003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.456020 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:29.456099 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.456103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.457014 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.457017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.458415 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.458427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.459307 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.459312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.459522 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.459571 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.459655 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.459746 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":421,"thread_start_us":114,"threads_started":1}
I20260806 04:34:29.460572 15522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a23a7c414e44b992ea434ee3b1f9db" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd" is_pre_election: true
I20260806 04:34:29.460731 15522 raft_consensus.cc:2454] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 0.
I20260806 04:34:29.460734 15095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a23a7c414e44b992ea434ee3b1f9db" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0818a1c8f3f0431d995da012284def3d" is_pre_election: true
I20260806 04:34:29.460978 15095 raft_consensus.cc:2454] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 0.
I20260806 04:34:29.461086 15211 leader_election.cc:302] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c75e0f9c16534072b93d8a703a5be416, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:29.461375 15639 raft_consensus.cc:2781] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:29.461445 15639 raft_consensus.cc:482] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:29.461493 15639 raft_consensus.cc:3037] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.462409 15639 raft_consensus.cc:504] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.462729 15639 leader_election.cc:288] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 election: Requested vote from peers 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.463068 15095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a23a7c414e44b992ea434ee3b1f9db" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0818a1c8f3f0431d995da012284def3d"
I20260806 04:34:29.463090 15522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a23a7c414e44b992ea434ee3b1f9db" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd"
I20260806 04:34:29.463165 15095 raft_consensus.cc:3037] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.463198 15522 raft_consensus.cc:3037] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.464126 15095 raft_consensus.cc:2454] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 1.
I20260806 04:34:29.464270 15522 raft_consensus.cc:2454] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 1.
I20260806 04:34:29.464401 15214 leader_election.cc:302] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0818a1c8f3f0431d995da012284def3d, c75e0f9c16534072b93d8a703a5be416; no voters:
I20260806 04:34:29.464576 15639 raft_consensus.cc:2781] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:29.464747 15639 raft_consensus.cc:686] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [term 1 LEADER]: Becoming Leader. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Running, Role: LEADER
I20260806 04:34:29.465065 15639 consensus_queue.cc:237] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.467227 15676 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: TxnStatusTablet state changed. Reason: New leader c75e0f9c16534072b93d8a703a5be416. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.467340 15676 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:29.467622 15678 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 04:34:29.467825 15678 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 04:34:29.468240 14865 catalog_manager.cc:5495] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75e0f9c16534072b93d8a703a5be416 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:29.508637 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.504817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.12.15.194:41877)
0806 04:34:29.504930 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.504933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.505009 (+ 76us) server_negotiation.cc:407] Connection header received
0806 04:34:29.505206 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.505208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.505263 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.505347 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:29.505577 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.506081 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.507251 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.507460 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.507537 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.507846 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.508264 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.508381 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.508471 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.508522 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 04:34:29.508694 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.504686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.0.0.1:42952)
0806 04:34:29.504939 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.504964 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.505079 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.505368 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.505371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.505378 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:29.505490 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.505496 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.506175 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.506179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.507142 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.507154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.507554 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.507558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.507652 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.507708 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.508369 (+ 661us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.508383 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.508462 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.508548 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 04:34:29.513718 15095 raft_consensus.cc:1261] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Refusing update from remote peer c75e0f9c16534072b93d8a703a5be416: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:29.513721 15522 raft_consensus.cc:1261] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Refusing update from remote peer c75e0f9c16534072b93d8a703a5be416: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:29.514416 15676 consensus_queue.cc:1035] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:29.514618 15639 consensus_queue.cc:1035] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:29.517655 15638 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: TxnStatusTablet state changed. Reason: New leader c75e0f9c16534072b93d8a703a5be416. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.517774 15638 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.520045 15643 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: TxnStatusTablet state changed. Reason: New leader c75e0f9c16534072b93d8a703a5be416. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.520156 15643 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.521739 15639 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.521845 15639 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:29.523008 15638 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.523118 15638 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P 0818a1c8f3f0431d995da012284def3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.523496 15685 mvcc.cc:205] Tried to move back new op lower bound from 7315418601521311744 to 7315418601331429376. Current Snapshot: MvccSnapshot[applied={T|T < 7315418601521311744}]
I20260806 04:34:29.524945 15676 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.525061 15676 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P c75e0f9c16534072b93d8a703a5be416: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:29.526916 15643 tablet_replica.cc:439] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } } }
I20260806 04:34:29.527032 15643 tablet_replica.cc:442] T 20a23a7c414e44b992ea434ee3b1f9db P cc971183b2034bfe9b700c80b84606fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:29.527962 15688 mvcc.cc:205] Tried to move back new op lower bound from 7315418601521311744 to 7315418601331429376. Current Snapshot: MvccSnapshot[applied={T|T < 7315418601521311744 or (T in {7315418601521311744})}]
I20260806 04:34:29.535205 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.531793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.12.15.193:35601)
0806 04:34:29.531903 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.531908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.531978 (+ 70us) server_negotiation.cc:407] Connection header received
0806 04:34:29.532159 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.532161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.532212 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.532297 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:29.532556 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.533055 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.533893 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.534115 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.534202 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.534462 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.534850 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.534949 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.535026 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.535078 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:34:29.546123 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.542520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.12.15.193:33627)
0806 04:34:29.542802 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.542814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.542909 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.543127 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.543129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.543134 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:29.543231 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.543234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.544208 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.544211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.544919 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.544930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.545469 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.545472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.545563 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.545607 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.545891 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.545895 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.545936 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.545993 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260806 04:34:29.546135 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.542647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:33627 (local address 127.12.15.194:41877)
0806 04:34:29.542773 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.542777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.542875 (+ 98us) server_negotiation.cc:407] Connection header received
0806 04:34:29.542994 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.542997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.543046 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.543125 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:29.543314 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.544111 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.545027 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.545366 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.545471 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.545739 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.545818 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.545886 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.545957 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.546008 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:34:29.552565 15522 raft_consensus.cc:1261] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Refusing update from remote peer 0818a1c8f3f0431d995da012284def3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:29.552897 15325 raft_consensus.cc:1261] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Refusing update from remote peer 0818a1c8f3f0431d995da012284def3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:29.553442 15648 consensus_queue.cc:1035] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:29.553635 15638 consensus_queue.cc:1035] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:29.558885 15683 mvcc.cc:205] Tried to move back new op lower bound from 7315418601682718720 to 7315418600463847424. Current Snapshot: MvccSnapshot[applied={T|T < 7315418601682718720}]
I20260806 04:34:29.575508 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.571470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:53749 (local address 127.12.15.193:35601)
0806 04:34:29.571584 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:29.571589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:29.571602 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:29.571890 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:29.571893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:29.571947 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:29.571994 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:29.572682 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.573327 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.574428 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.574658 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.574749 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.575037 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.575110 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.575167 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:29.575263 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 04:34:29.575315 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 04:34:29.575668 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:29.571201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35601 (local address 127.12.15.194:53749)
0806 04:34:29.571505 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:29.571520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:29.571764 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:29.572405 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:29.572408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:29.572417 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:29.572548 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.572554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.573573 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.573576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.574318 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:29.574329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:29.574783 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:29.574787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:29.574868 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:29.574914 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:29.575182 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:29.575189 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:34:29.575243 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 04:34:29.575326 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":92,"threads_started":1}
I20260806 04:34:29.604090 14865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:29.604328 14865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:29.607278 14865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Sending DeleteTablet for 3 replicas of tablet 658cc2e71d2a4e72b860511c90cc1d1e
I20260806 04:34:29.608039 15292 tablet_service.cc:1514] Processing DeleteTablet for tablet 658cc2e71d2a4e72b860511c90cc1d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:29 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:42946
I20260806 04:34:29.608078 15491 tablet_service.cc:1514] Processing DeleteTablet for tablet 658cc2e71d2a4e72b860511c90cc1d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:29 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45718
I20260806 04:34:29.608729 15711 tablet_replica.cc:330] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: stopping tablet replica
I20260806 04:34:29.608966 15712 tablet_replica.cc:330] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: stopping tablet replica
I20260806 04:34:29.608935 15071 tablet_service.cc:1514] Processing DeleteTablet for tablet 658cc2e71d2a4e72b860511c90cc1d1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:29 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:55946
I20260806 04:34:29.609331 15712 raft_consensus.cc:2229] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:29.609673 15711 raft_consensus.cc:2229] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:29.609894 15711 raft_consensus.cc:2258] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:29.610148 15712 raft_consensus.cc:2258] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:29.610459 14871 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:29.610901 14871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:29.611861 15713 tablet_replica.cc:330] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: stopping tablet replica
I20260806 04:34:29.612133 15713 raft_consensus.cc:2229] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:29.612517 15713 raft_consensus.cc:2258] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:29.614051 15713 ts_tablet_manager.cc:1905] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:29.615999 15071 tablet_service.cc:1467] Processing CreateTablet for tablet e35b98553db44497b1bc36ed38a7295e (DEFAULT_TABLE table=test-table [id=88dce3a7b56248628963aa6587c21541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:29.616287 15071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e35b98553db44497b1bc36ed38a7295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.617239 15713 ts_tablet_manager.cc:1918] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:29.617324 15713 log.cc:1196] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/658cc2e71d2a4e72b860511c90cc1d1e
I20260806 04:34:29.617622 15713 ts_tablet_manager.cc:1939] T 658cc2e71d2a4e72b860511c90cc1d1e P 0818a1c8f3f0431d995da012284def3d: Deleting consensus metadata
I20260806 04:34:29.618681 15491 tablet_service.cc:1467] Processing CreateTablet for tablet e35b98553db44497b1bc36ed38a7295e (DEFAULT_TABLE table=test-table [id=88dce3a7b56248628963aa6587c21541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:29.618922 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e35b98553db44497b1bc36ed38a7295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.620510 15630 tablet_bootstrap.cc:492] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Bootstrap starting.
I20260806 04:34:29.621209 15634 tablet_bootstrap.cc:492] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Bootstrap starting.
I20260806 04:34:29.621444 15630 tablet_bootstrap.cc:654] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.622123 15634 tablet_bootstrap.cc:654] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.623081 15292 tablet_service.cc:1467] Processing CreateTablet for tablet e35b98553db44497b1bc36ed38a7295e (DEFAULT_TABLE table=test-table [id=88dce3a7b56248628963aa6587c21541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:29.623358 15292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e35b98553db44497b1bc36ed38a7295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:29.624471 15711 ts_tablet_manager.cc:1905] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:29.624329 15712 ts_tablet_manager.cc:1905] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:29.626086 14851 catalog_manager.cc:4841] TS 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601): tablet 658cc2e71d2a4e72b860511c90cc1d1e (table test-table [id=4621eb1d170d4bf1b43d3c2ab5ec3ab5]) successfully deleted
I20260806 04:34:29.625834 15631 tablet_bootstrap.cc:492] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Bootstrap starting.
I20260806 04:34:29.627103 15631 tablet_bootstrap.cc:654] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:29.629493 15630 tablet_bootstrap.cc:492] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: No bootstrap required, opened a new log
I20260806 04:34:29.629585 15630 ts_tablet_manager.cc:1397] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260806 04:34:29.630045 15630 raft_consensus.cc:348] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.630160 15630 raft_consensus.cc:374] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.630192 15630 raft_consensus.cc:729] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.630309 15630 consensus_queue.cc:260] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.630575 15630 ts_tablet_manager.cc:1428] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:29.630755 15711 ts_tablet_manager.cc:1918] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:29.630927 15712 ts_tablet_manager.cc:1918] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:29.631094 15711 log.cc:1196] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/658cc2e71d2a4e72b860511c90cc1d1e
I20260806 04:34:29.631224 15712 log.cc:1196] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/658cc2e71d2a4e72b860511c90cc1d1e
I20260806 04:34:29.631594 15711 ts_tablet_manager.cc:1939] T 658cc2e71d2a4e72b860511c90cc1d1e P c75e0f9c16534072b93d8a703a5be416: Deleting consensus metadata
I20260806 04:34:29.631969 15712 ts_tablet_manager.cc:1939] T 658cc2e71d2a4e72b860511c90cc1d1e P cc971183b2034bfe9b700c80b84606fd: Deleting consensus metadata
I20260806 04:34:29.632382 15631 tablet_bootstrap.cc:492] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: No bootstrap required, opened a new log
I20260806 04:34:29.632467 15631 ts_tablet_manager.cc:1397] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260806 04:34:29.632836 15631 raft_consensus.cc:348] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.632957 15631 raft_consensus.cc:374] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.632992 15631 raft_consensus.cc:729] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.633100 15631 consensus_queue.cc:260] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.633332 15631 ts_tablet_manager.cc:1428] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:29.634259 15634 tablet_bootstrap.cc:492] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: No bootstrap required, opened a new log
I20260806 04:34:29.634342 15634 ts_tablet_manager.cc:1397] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260806 04:34:29.634908 14850 catalog_manager.cc:4841] TS c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877): tablet 658cc2e71d2a4e72b860511c90cc1d1e (table test-table [id=4621eb1d170d4bf1b43d3c2ab5ec3ab5]) successfully deleted
I20260806 04:34:29.635020 15634 raft_consensus.cc:348] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.635141 15634 raft_consensus.cc:374] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:29.635174 15634 raft_consensus.cc:729] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc971183b2034bfe9b700c80b84606fd, State: Initialized, Role: FOLLOWER
I20260806 04:34:29.635242 14848 catalog_manager.cc:4841] TS cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009): tablet 658cc2e71d2a4e72b860511c90cc1d1e (table test-table [id=4621eb1d170d4bf1b43d3c2ab5ec3ab5]) successfully deleted
I20260806 04:34:29.635294 15634 consensus_queue.cc:260] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.635531 15634 ts_tablet_manager.cc:1428] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:29.688123 15638 raft_consensus.cc:482] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:29.688268 15638 raft_consensus.cc:504] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.688753 15638 leader_election.cc:288] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.689283 15521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e35b98553db44497b1bc36ed38a7295e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd" is_pre_election: true
I20260806 04:34:29.689277 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e35b98553db44497b1bc36ed38a7295e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75e0f9c16534072b93d8a703a5be416" is_pre_election: true
I20260806 04:34:29.689445 15521 raft_consensus.cc:2454] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 0.
I20260806 04:34:29.689445 15325 raft_consensus.cc:2454] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 0.
I20260806 04:34:29.689777 15002 leader_election.cc:302] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0818a1c8f3f0431d995da012284def3d, c75e0f9c16534072b93d8a703a5be416; no voters:
I20260806 04:34:29.689965 15638 raft_consensus.cc:2781] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:29.690048 15638 raft_consensus.cc:482] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:29.690088 15638 raft_consensus.cc:3037] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.691274 15638 raft_consensus.cc:504] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.691644 15638 leader_election.cc:288] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 election: Requested vote from peers c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:29.692070 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e35b98553db44497b1bc36ed38a7295e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75e0f9c16534072b93d8a703a5be416"
I20260806 04:34:29.692095 15521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e35b98553db44497b1bc36ed38a7295e" candidate_uuid: "0818a1c8f3f0431d995da012284def3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd"
I20260806 04:34:29.692209 15325 raft_consensus.cc:3037] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.692215 15521 raft_consensus.cc:3037] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:29.693220 15521 raft_consensus.cc:2454] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 1.
I20260806 04:34:29.693364 15325 raft_consensus.cc:2454] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0818a1c8f3f0431d995da012284def3d in term 1.
I20260806 04:34:29.693495 15001 leader_election.cc:302] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0818a1c8f3f0431d995da012284def3d, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:29.693670 15638 raft_consensus.cc:2781] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:29.693765 15638 raft_consensus.cc:686] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 1 LEADER]: Becoming Leader. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Running, Role: LEADER
I20260806 04:34:29.693898 15638 consensus_queue.cc:237] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:29.695710 14865 catalog_manager.cc:5495] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0818a1c8f3f0431d995da012284def3d (127.12.15.193). New cstate: current_term: 1 leader_uuid: "0818a1c8f3f0431d995da012284def3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:29.724962 15521 raft_consensus.cc:1261] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Refusing update from remote peer 0818a1c8f3f0431d995da012284def3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:29.725075 15325 raft_consensus.cc:1261] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Refusing update from remote peer 0818a1c8f3f0431d995da012284def3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:29.725507 15648 consensus_queue.cc:1035] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:29.725806 15648 consensus_queue.cc:1035] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:34:29.744380 15685 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 04:34:31.298240 14871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:31.298393 14871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:31.300699 14871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Sending DeleteTablet for 3 replicas of tablet e35b98553db44497b1bc36ed38a7295e
I20260806 04:34:31.301183 15071 tablet_service.cc:1514] Processing DeleteTablet for tablet e35b98553db44497b1bc36ed38a7295e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:31 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:55946
I20260806 04:34:31.301326 15292 tablet_service.cc:1514] Processing DeleteTablet for tablet e35b98553db44497b1bc36ed38a7295e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:31 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:42946
I20260806 04:34:31.301420 15491 tablet_service.cc:1514] Processing DeleteTablet for tablet e35b98553db44497b1bc36ed38a7295e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:31 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45718
I20260806 04:34:31.301712 15754 tablet_replica.cc:330] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: stopping tablet replica
I20260806 04:34:31.301810 15755 tablet_replica.cc:330] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: stopping tablet replica
I20260806 04:34:31.301882 15754 raft_consensus.cc:2229] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:31.301941 15755 raft_consensus.cc:2229] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:31.302055 15755 raft_consensus.cc:2258] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:31.302067 15754 raft_consensus.cc:2258] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:31.302742 15755 ts_tablet_manager.cc:1905] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:31.302742 15754 ts_tablet_manager.cc:1905] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:31.305493 15755 ts_tablet_manager.cc:1918] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:34:31.305493 15754 ts_tablet_manager.cc:1918] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:34:31.305577 15754 log.cc:1196] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/e35b98553db44497b1bc36ed38a7295e
I20260806 04:34:31.305577 15755 log.cc:1196] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/e35b98553db44497b1bc36ed38a7295e
I20260806 04:34:31.305845 15755 ts_tablet_manager.cc:1939] T e35b98553db44497b1bc36ed38a7295e P c75e0f9c16534072b93d8a703a5be416: Deleting consensus metadata
I20260806 04:34:31.305884 15754 ts_tablet_manager.cc:1939] T e35b98553db44497b1bc36ed38a7295e P 0818a1c8f3f0431d995da012284def3d: Deleting consensus metadata
I20260806 04:34:31.306155 15756 tablet_replica.cc:330] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: stopping tablet replica
I20260806 04:34:31.306291 15756 raft_consensus.cc:2229] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:31.306412 15756 raft_consensus.cc:2258] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:31.306494 14851 catalog_manager.cc:4841] TS 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601): tablet e35b98553db44497b1bc36ed38a7295e (table test-table [id=88dce3a7b56248628963aa6587c21541]) successfully deleted
I20260806 04:34:31.306798 14850 catalog_manager.cc:4841] TS c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877): tablet e35b98553db44497b1bc36ed38a7295e (table test-table [id=88dce3a7b56248628963aa6587c21541]) successfully deleted
I20260806 04:34:31.307250 15756 ts_tablet_manager.cc:1905] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:31.309547 15756 ts_tablet_manager.cc:1918] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:34:31.309633 15756 log.cc:1196] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/e35b98553db44497b1bc36ed38a7295e
I20260806 04:34:31.309901 15756 ts_tablet_manager.cc:1939] T e35b98553db44497b1bc36ed38a7295e P cc971183b2034bfe9b700c80b84606fd: Deleting consensus metadata
I20260806 04:34:31.310611 14848 catalog_manager.cc:4841] TS cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009): tablet e35b98553db44497b1bc36ed38a7295e (table test-table [id=88dce3a7b56248628963aa6587c21541]) successfully deleted
Aug 06 04:34:31 dist-test-slave-xt4k krb5kdc[14807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:34:31 dist-test-slave-xt4k krb5kdc[14807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:31.329890 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.321760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.12.15.254:45285)
0806 04:34:31.322004 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.322011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.322028 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:34:31.322150 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.322152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.322198 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.322278 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:31.322916 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.323455 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.324424 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.324669 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.324758 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.327041 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:31.327044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:31.327046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:31.327069 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:31.328489 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:31.328893 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:31.328895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:31.328897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:31.328970 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:31.329220 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:31.329222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:31.329223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:31.329389 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:31.329484 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.329661 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.329758 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260806 04:34:31.332542 14871 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:31.333014 14871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:31.337674 15071 tablet_service.cc:1467] Processing CreateTablet for tablet b87bce003915403293002d96cc71afe0 (DEFAULT_TABLE table=test-table [id=1504be68ab1e4f159d4c416add28e421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:31.337879 15491 tablet_service.cc:1467] Processing CreateTablet for tablet b87bce003915403293002d96cc71afe0 (DEFAULT_TABLE table=test-table [id=1504be68ab1e4f159d4c416add28e421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:31.337994 15071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87bce003915403293002d96cc71afe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:31.337919 15292 tablet_service.cc:1467] Processing CreateTablet for tablet b87bce003915403293002d96cc71afe0 (DEFAULT_TABLE table=test-table [id=1504be68ab1e4f159d4c416add28e421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:31.338150 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87bce003915403293002d96cc71afe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:31.338227 15292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87bce003915403293002d96cc71afe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:31.340636 15772 tablet_bootstrap.cc:492] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Bootstrap starting.
I20260806 04:34:31.340932 15773 tablet_bootstrap.cc:492] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Bootstrap starting.
I20260806 04:34:31.341050 15771 tablet_bootstrap.cc:492] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Bootstrap starting.
I20260806 04:34:31.341635 15772 tablet_bootstrap.cc:654] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:31.341770 15773 tablet_bootstrap.cc:654] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:31.341886 15771 tablet_bootstrap.cc:654] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:31.342887 15772 tablet_bootstrap.cc:492] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: No bootstrap required, opened a new log
I20260806 04:34:31.342960 15771 tablet_bootstrap.cc:492] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: No bootstrap required, opened a new log
I20260806 04:34:31.343004 15773 tablet_bootstrap.cc:492] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: No bootstrap required, opened a new log
I20260806 04:34:31.343012 15772 ts_tablet_manager.cc:1397] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:31.343043 15771 ts_tablet_manager.cc:1397] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:31.343080 15773 ts_tablet_manager.cc:1397] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:31.343446 15772 raft_consensus.cc:348] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.343446 15773 raft_consensus.cc:348] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.343479 15771 raft_consensus.cc:348] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.343555 15772 raft_consensus.cc:374] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:31.343565 15773 raft_consensus.cc:374] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:31.343590 15771 raft_consensus.cc:374] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:31.343590 15772 raft_consensus.cc:729] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc971183b2034bfe9b700c80b84606fd, State: Initialized, Role: FOLLOWER
I20260806 04:34:31.343596 15773 raft_consensus.cc:729] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Initialized, Role: FOLLOWER
I20260806 04:34:31.343624 15771 raft_consensus.cc:729] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Initialized, Role: FOLLOWER
I20260806 04:34:31.343716 15772 consensus_queue.cc:260] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.343724 15773 consensus_queue.cc:260] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.343770 15771 consensus_queue.cc:260] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.343986 15773 ts_tablet_manager.cc:1428] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:31.344233 15772 ts_tablet_manager.cc:1428] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:31.344447 15771 ts_tablet_manager.cc:1428] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:31.449244 15777 raft_consensus.cc:482] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:31.449402 15777 raft_consensus.cc:504] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.450591 15777 leader_election.cc:288] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601), c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877)
I20260806 04:34:31.454701 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.450939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:59757 (local address 127.12.15.194:41877)
0806 04:34:31.451272 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.451281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.451427 (+ 146us) server_negotiation.cc:407] Connection header received
0806 04:34:31.451574 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.451576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.451627 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.451713 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:31.451954 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.452554 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.453661 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.454104 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.454221 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.454274 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.454489 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.454571 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":133,"threads_started":1}
I20260806 04:34:31.454713 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.450853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35601 (local address 127.12.15.195:34863)
0806 04:34:31.451269 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:31.451284 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:31.451394 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:31.451638 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:31.451641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:31.451659 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:31.451785 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.451793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.452602 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.452605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.453535 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.453544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.454221 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.454225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.454376 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.454394 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 04:34:31.454457 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 04:34:31.454530 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":55,"thread_start_us":162,"threads_started":1}
I20260806 04:34:31.454701 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.450937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:34863 (local address 127.12.15.193:35601)
0806 04:34:31.451377 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.451386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.451403 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:34:31.451487 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.451491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.451541 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.451613 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:31.451912 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.452500 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.453655 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.454086 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.454192 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.454251 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.454492 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.454567 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":136,"threads_started":1}
I20260806 04:34:31.454723 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.450853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.12.15.195:59757)
0806 04:34:31.451329 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:31.451343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:31.451470 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:31.451736 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:31.451738 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:31.451751 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:31.451848 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.451854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.452680 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.452683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.453536 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.453545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.454220 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.454224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.454371 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.454391 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:34:31.454463 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 04:34:31.454533 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":94,"spinlock_wait_cycles":31104,"thread_start_us":207,"threads_started":1}
I20260806 04:34:31.455171 15095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bce003915403293002d96cc71afe0" candidate_uuid: "cc971183b2034bfe9b700c80b84606fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0818a1c8f3f0431d995da012284def3d" is_pre_election: true
I20260806 04:34:31.455332 15095 raft_consensus.cc:2454] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc971183b2034bfe9b700c80b84606fd in term 0.
I20260806 04:34:31.455579 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bce003915403293002d96cc71afe0" candidate_uuid: "cc971183b2034bfe9b700c80b84606fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75e0f9c16534072b93d8a703a5be416" is_pre_election: true
I20260806 04:34:31.455711 15325 raft_consensus.cc:2454] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc971183b2034bfe9b700c80b84606fd in term 0.
I20260806 04:34:31.455709 15430 leader_election.cc:302] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0818a1c8f3f0431d995da012284def3d, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:31.455994 15777 raft_consensus.cc:2781] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:31.456058 15777 raft_consensus.cc:482] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:31.456100 15777 raft_consensus.cc:3037] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:31.457147 15777 raft_consensus.cc:504] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.457440 15777 leader_election.cc:288] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [CANDIDATE]: Term 1 election: Requested vote from peers 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601), c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877)
I20260806 04:34:31.457806 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bce003915403293002d96cc71afe0" candidate_uuid: "cc971183b2034bfe9b700c80b84606fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75e0f9c16534072b93d8a703a5be416"
I20260806 04:34:31.457916 15325 raft_consensus.cc:3037] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:31.458823 15325 raft_consensus.cc:2454] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc971183b2034bfe9b700c80b84606fd in term 1.
I20260806 04:34:31.459156 15095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87bce003915403293002d96cc71afe0" candidate_uuid: "cc971183b2034bfe9b700c80b84606fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0818a1c8f3f0431d995da012284def3d"
I20260806 04:34:31.459173 15429 leader_election.cc:302] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c75e0f9c16534072b93d8a703a5be416, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:31.459273 15095 raft_consensus.cc:3037] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:31.459363 15777 raft_consensus.cc:2781] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:31.459601 15777 raft_consensus.cc:686] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 1 LEADER]: Becoming Leader. State: Replica: cc971183b2034bfe9b700c80b84606fd, State: Running, Role: LEADER
I20260806 04:34:31.459869 15777 consensus_queue.cc:237] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.460255 15095 raft_consensus.cc:2454] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc971183b2034bfe9b700c80b84606fd in term 1.
I20260806 04:34:31.462374 14875 catalog_manager.cc:5495] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd reported cstate change: term changed from 0 to 1, leader changed from <none> to cc971183b2034bfe9b700c80b84606fd (127.12.15.195). New cstate: current_term: 1 leader_uuid: "cc971183b2034bfe9b700c80b84606fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } health_report { overall_health: HEALTHY } } }
I20260806 04:34:31.501765 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.498025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.12.15.195:41009)
0806 04:34:31.498298 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.498304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.498315 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:34:31.498386 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.498388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.498438 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.498541 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:31.498790 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.499364 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.500203 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.500475 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.500595 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.500850 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.501339 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.501442 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.501527 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.501583 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":134,"threads_started":1}
I20260806 04:34:31.511422 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.508587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:56043 (local address 127.12.15.194:41877)
0806 04:34:31.508704 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.508708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.508860 (+ 152us) server_negotiation.cc:407] Connection header received
0806 04:34:31.508955 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.508957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.509008 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.509071 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:31.509262 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.509745 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.510570 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.510781 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.510866 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.511090 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.511155 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.511200 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.511267 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.511319 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:34:31.511437 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.508484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.12.15.195:56043)
0806 04:34:31.508794 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:31.508806 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:31.508889 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:31.509079 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:31.509081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:31.509086 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:31.509190 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.509194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.509822 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.509824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.510483 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.510491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.510874 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.510877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.510950 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.510992 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.511209 (+ 217us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.511212 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 04:34:31.511253 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 04:34:31.511312 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":129,"threads_started":1}
I20260806 04:34:31.517204 15325 raft_consensus.cc:1261] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Refusing update from remote peer cc971183b2034bfe9b700c80b84606fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:31.517333 15095 raft_consensus.cc:1261] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Refusing update from remote peer cc971183b2034bfe9b700c80b84606fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:31.517654 15777 consensus_queue.cc:1035] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:31.517823 15785 consensus_queue.cc:1035] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:31.535080 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.531890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:60753 (local address 127.12.15.195:41009)
0806 04:34:31.532008 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.532012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.532175 (+ 163us) server_negotiation.cc:407] Connection header received
0806 04:34:31.532311 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.532314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.532388 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.532457 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:31.532740 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.533260 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.534033 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.534313 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.534391 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.534667 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.534753 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.534819 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.534904 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.534962 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 04:34:31.535112 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.531852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41009 (local address 127.12.15.194:60753)
0806 04:34:31.532100 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:31.532114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:31.532219 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:31.532480 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:31.532483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:31.532499 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:31.532629 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.532638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.533341 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.533343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.533939 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.533946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.534403 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.534407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.534527 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.534557 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.534837 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.534843 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:34:31.534890 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 04:34:31.534970 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260806 04:34:31.557827 14865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:31.557977 14865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:31.559988 14865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Sending DeleteTablet for 3 replicas of tablet b87bce003915403293002d96cc71afe0
I20260806 04:34:31.560591 15071 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bce003915403293002d96cc71afe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:31 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:55946
I20260806 04:34:31.560783 15292 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bce003915403293002d96cc71afe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:31 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:42946
I20260806 04:34:31.560968 15755 tablet_replica.cc:330] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: stopping tablet replica
I20260806 04:34:31.560979 15754 tablet_replica.cc:330] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: stopping tablet replica
I20260806 04:34:31.561087 15755 raft_consensus.cc:2229] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:31.561110 15754 raft_consensus.cc:2229] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:31.561223 15755 raft_consensus.cc:2258] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:31.561223 15754 raft_consensus.cc:2258] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:31.561200 14865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 04:34:31.561511 15491 tablet_service.cc:1514] Processing DeleteTablet for tablet b87bce003915403293002d96cc71afe0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:31 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45718
W20260806 04:34:31.561596 14865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:31.561668 15756 tablet_replica.cc:330] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: stopping tablet replica
I20260806 04:34:31.561766 15756 raft_consensus.cc:2229] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:31.562050 15756 raft_consensus.cc:2258] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:31.562083 15754 ts_tablet_manager.cc:1905] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:31.563059 15756 ts_tablet_manager.cc:1905] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:31.565371 15756 ts_tablet_manager.cc:1918] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:31.565423 15756 log.cc:1196] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/b87bce003915403293002d96cc71afe0
I20260806 04:34:31.565615 15756 ts_tablet_manager.cc:1939] T b87bce003915403293002d96cc71afe0 P cc971183b2034bfe9b700c80b84606fd: Deleting consensus metadata
I20260806 04:34:31.566258 14848 catalog_manager.cc:4841] TS cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009): tablet b87bce003915403293002d96cc71afe0 (table test-table [id=1504be68ab1e4f159d4c416add28e421]) successfully deleted
I20260806 04:34:31.566349 15754 ts_tablet_manager.cc:1918] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:31.566421 15754 log.cc:1196] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/b87bce003915403293002d96cc71afe0
I20260806 04:34:31.566713 15754 ts_tablet_manager.cc:1939] T b87bce003915403293002d96cc71afe0 P 0818a1c8f3f0431d995da012284def3d: Deleting consensus metadata
I20260806 04:34:31.567180 15491 tablet_service.cc:1467] Processing CreateTablet for tablet b6b0b891241f412a9882c3b22ac599d1 (DEFAULT_TABLE table=test-table [id=f6870da40898436b9245bc43cd21ba34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:31.567229 15071 tablet_service.cc:1467] Processing CreateTablet for tablet b6b0b891241f412a9882c3b22ac599d1 (DEFAULT_TABLE table=test-table [id=f6870da40898436b9245bc43cd21ba34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:31.567425 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b0b891241f412a9882c3b22ac599d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:31.567377 15292 tablet_service.cc:1467] Processing CreateTablet for tablet b6b0b891241f412a9882c3b22ac599d1 (DEFAULT_TABLE table=test-table [id=f6870da40898436b9245bc43cd21ba34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:31.567479 15071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b0b891241f412a9882c3b22ac599d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:31.567618 15292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6b0b891241f412a9882c3b22ac599d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:31.569490 15772 tablet_bootstrap.cc:492] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Bootstrap starting.
I20260806 04:34:31.569797 15773 tablet_bootstrap.cc:492] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d: Bootstrap starting.
I20260806 04:34:31.570030 15771 tablet_bootstrap.cc:492] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Bootstrap starting.
I20260806 04:34:31.570431 15772 tablet_bootstrap.cc:654] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:31.570564 15773 tablet_bootstrap.cc:654] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:31.570924 15771 tablet_bootstrap.cc:654] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:31.571859 15773 tablet_bootstrap.cc:492] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d: No bootstrap required, opened a new log
I20260806 04:34:31.571941 15773 ts_tablet_manager.cc:1397] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:31.572319 15771 tablet_bootstrap.cc:492] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: No bootstrap required, opened a new log
I20260806 04:34:31.572425 15771 ts_tablet_manager.cc:1397] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:31.572324 15773 raft_consensus.cc:348] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.572577 15773 raft_consensus.cc:374] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:31.572616 15773 raft_consensus.cc:729] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0818a1c8f3f0431d995da012284def3d, State: Initialized, Role: FOLLOWER
I20260806 04:34:31.572762 15773 consensus_queue.cc:260] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.572817 15771 raft_consensus.cc:348] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.572927 15771 raft_consensus.cc:374] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:31.572966 15771 raft_consensus.cc:729] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Initialized, Role: FOLLOWER
I20260806 04:34:31.573031 15773 ts_tablet_manager.cc:1428] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 04:34:31.573087 15771 consensus_queue.cc:260] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.573347 15771 ts_tablet_manager.cc:1428] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 04:34:31.574035 14851 catalog_manager.cc:4841] TS 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601): tablet b87bce003915403293002d96cc71afe0 (table test-table [id=1504be68ab1e4f159d4c416add28e421]) successfully deleted
I20260806 04:34:31.574703 15755 ts_tablet_manager.cc:1905] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:31.576033 15772 tablet_bootstrap.cc:492] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: No bootstrap required, opened a new log
I20260806 04:34:31.576128 15772 ts_tablet_manager.cc:1397] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260806 04:34:31.576644 15772 raft_consensus.cc:348] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.576756 15772 raft_consensus.cc:374] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:31.576784 15772 raft_consensus.cc:729] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc971183b2034bfe9b700c80b84606fd, State: Initialized, Role: FOLLOWER
I20260806 04:34:31.576893 15772 consensus_queue.cc:260] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.577147 15772 ts_tablet_manager.cc:1428] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 04:34:31.577356 15755 ts_tablet_manager.cc:1918] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:31.577430 15755 log.cc:1196] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/b87bce003915403293002d96cc71afe0
I20260806 04:34:31.577641 15755 ts_tablet_manager.cc:1939] T b87bce003915403293002d96cc71afe0 P c75e0f9c16534072b93d8a703a5be416: Deleting consensus metadata
I20260806 04:34:31.578239 14850 catalog_manager.cc:4841] TS c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877): tablet b87bce003915403293002d96cc71afe0 (table test-table [id=1504be68ab1e4f159d4c416add28e421]) successfully deleted
I20260806 04:34:31.583372 15639 raft_consensus.cc:482] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:31.583473 15639 raft_consensus.cc:504] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.583868 15639 leader_election.cc:288] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:31.584194 15521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0b891241f412a9882c3b22ac599d1" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd" is_pre_election: true
I20260806 04:34:31.584354 15521 raft_consensus.cc:2454] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 0.
I20260806 04:34:31.584324 15095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0b891241f412a9882c3b22ac599d1" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0818a1c8f3f0431d995da012284def3d" is_pre_election: true
I20260806 04:34:31.584481 15095 raft_consensus.cc:2454] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 0.
I20260806 04:34:31.584692 15211 leader_election.cc:302] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c75e0f9c16534072b93d8a703a5be416, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:31.584851 15639 raft_consensus.cc:2781] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:31.584915 15639 raft_consensus.cc:482] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:31.584960 15639 raft_consensus.cc:3037] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:31.585774 15639 raft_consensus.cc:504] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.586073 15639 leader_election.cc:288] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 election: Requested vote from peers 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601), cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009)
I20260806 04:34:31.586404 15095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0b891241f412a9882c3b22ac599d1" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0818a1c8f3f0431d995da012284def3d"
I20260806 04:34:31.586419 15521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6b0b891241f412a9882c3b22ac599d1" candidate_uuid: "c75e0f9c16534072b93d8a703a5be416" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc971183b2034bfe9b700c80b84606fd"
I20260806 04:34:31.586524 15521 raft_consensus.cc:3037] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:31.586519 15095 raft_consensus.cc:3037] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:31.587378 15095 raft_consensus.cc:2454] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 1.
I20260806 04:34:31.587723 15521 raft_consensus.cc:2454] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75e0f9c16534072b93d8a703a5be416 in term 1.
I20260806 04:34:31.588044 15211 leader_election.cc:302] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c75e0f9c16534072b93d8a703a5be416, cc971183b2034bfe9b700c80b84606fd; no voters:
I20260806 04:34:31.588254 15639 raft_consensus.cc:2781] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:31.588411 15639 raft_consensus.cc:686] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 1 LEADER]: Becoming Leader. State: Replica: c75e0f9c16534072b93d8a703a5be416, State: Running, Role: LEADER
I20260806 04:34:31.588557 15639 consensus_queue.cc:237] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } }
I20260806 04:34:31.590042 14865 catalog_manager.cc:5495] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75e0f9c16534072b93d8a703a5be416 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "c75e0f9c16534072b93d8a703a5be416" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75e0f9c16534072b93d8a703a5be416" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 41877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:31.605419 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.602147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.12.15.194:41877)
0806 04:34:31.602258 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.602263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.602277 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:31.602377 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.602380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.602446 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.602523 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:31.602722 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.603499 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.604291 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.604654 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.604753 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.604950 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.605030 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.605093 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.605234 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.605287 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:34:31.613672 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.610869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:41877 (local address 127.12.15.194:52019)
0806 04:34:31.610991 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:31.611005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:31.611093 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:31.611283 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:31.611286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:31.611292 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:31.611363 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.611368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.612056 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.612059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.612672 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:31.612681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.613056 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.613059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:31.613141 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.613163 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.613384 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.613391 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:34:31.613447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 04:34:31.613565 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":17}
I20260806 04:34:31.613672 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:31.611029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:52019 (local address 127.12.15.194:41877)
0806 04:34:31.611130 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:31.611134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:31.611146 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:34:31.611178 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:31.611181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:31.611215 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:31.611270 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:31.611459 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.611959 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.612773 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:31.612966 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:31.613048 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:31.613256 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:31.613316 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:31.613373 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:31.613455 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 04:34:31.613535 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":41}
I20260806 04:34:31.619314 15095 raft_consensus.cc:1261] T b6b0b891241f412a9882c3b22ac599d1 P 0818a1c8f3f0431d995da012284def3d [term 1 FOLLOWER]: Refusing update from remote peer c75e0f9c16534072b93d8a703a5be416: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:31.619323 15521 raft_consensus.cc:1261] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Refusing update from remote peer c75e0f9c16534072b93d8a703a5be416: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:31.619722 15639 consensus_queue.cc:1035] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc971183b2034bfe9b700c80b84606fd" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:31.619997 15729 consensus_queue.cc:1035] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0818a1c8f3f0431d995da012284def3d" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:34:31.634423 15725 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 04:34:33.194187 14865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:33.194332 14865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:33.196576 14865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a3603ffb7164c9ca750af1573ae0a5c: Sending DeleteTablet for 3 replicas of tablet b6b0b891241f412a9882c3b22ac599d1
I20260806 04:34:33.197075 15292 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b0b891241f412a9882c3b22ac599d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:33 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:42946
I20260806 04:34:33.197141 15491 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b0b891241f412a9882c3b22ac599d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:33 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45718
I20260806 04:34:33.197407 15071 tablet_service.cc:1514] Processing DeleteTablet for tablet b6b0b891241f412a9882c3b22ac599d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:33 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:55946
I20260806 04:34:33.197413 15849 tablet_replica.cc:330] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: stopping tablet replica
I20260806 04:34:33.197551 15849 raft_consensus.cc:2229] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:33.197546 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14977
I20260806 04:34:33.207592 15850 tablet_replica.cc:330] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: stopping tablet replica
I20260806 04:34:33.207780 15850 raft_consensus.cc:2229] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:33.207928 15850 raft_consensus.cc:2258] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:33.208446 15849 raft_consensus.cc:2258] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:33.208822 15850 ts_tablet_manager.cc:1905] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:33.209118 15849 ts_tablet_manager.cc:1905] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:33.211719 15850 ts_tablet_manager.cc:1918] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:34:33.211719 15849 ts_tablet_manager.cc:1918] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 04:34:33.211807 15849 log.cc:1196] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/b6b0b891241f412a9882c3b22ac599d1
I20260806 04:34:33.211807 15850 log.cc:1196] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/b6b0b891241f412a9882c3b22ac599d1
I20260806 04:34:33.212076 15850 ts_tablet_manager.cc:1939] T b6b0b891241f412a9882c3b22ac599d1 P cc971183b2034bfe9b700c80b84606fd: Deleting consensus metadata
I20260806 04:34:33.212242 15849 ts_tablet_manager.cc:1939] T b6b0b891241f412a9882c3b22ac599d1 P c75e0f9c16534072b93d8a703a5be416: Deleting consensus metadata
I20260806 04:34:33.212965 14850 catalog_manager.cc:4841] TS c75e0f9c16534072b93d8a703a5be416 (127.12.15.194:41877): tablet b6b0b891241f412a9882c3b22ac599d1 (table test-table [id=f6870da40898436b9245bc43cd21ba34]) successfully deleted
I20260806 04:34:33.212983 14848 catalog_manager.cc:4841] TS cc971183b2034bfe9b700c80b84606fd (127.12.15.195:41009): tablet b6b0b891241f412a9882c3b22ac599d1 (table test-table [id=f6870da40898436b9245bc43cd21ba34]) successfully deleted
W20260806 04:34:33.215045 14851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:35601 (error 108)
I20260806 04:34:33.215145 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 15186
I20260806 04:34:33.225505 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:33.225027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35601 (local address 127.0.0.1:55958)
0806 04:34:33.225312 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:33.225392 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:35601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
W20260806 04:34:33.225659 14851 catalog_manager.cc:4584] TS 0818a1c8f3f0431d995da012284def3d (127.12.15.193:35601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6b0b891241f412a9882c3b22ac599d1: Network error: Client connection negotiation failed: client connection to 127.12.15.193:35601: connect: Connection refused (error 111)
I20260806 04:34:33.230609 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 15403
I20260806 04:34:33.245167 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14824
2026-08-06T04:34:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6335 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:33 dist-test-slave-xt4k krb5kdc[15857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:33 dist-test-slave-xt4k krb5kdc[15857](info): set up 2 sockets
Aug 06 04:34:33 dist-test-slave-xt4k krb5kdc[15857](info): commencing operation
krb5kdc: starting...
W20260806 04:34:34.391692 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:34 dist-test-slave-xt4k krb5kdc[15857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:34.516867 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36665
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:34.628988 15878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:34.629271 15878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:34.629310 15878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:34.633292 15878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:34.633349 15878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:34.633368 15878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:34.633383 15878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:34.633395 15878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:34.638660 15878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:34.639691 15878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:34.640730 15878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:34.649327 15885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:34.649438 15878 server_base.cc:1034] running on GCE node
W20260806 04:34:34.649314 15888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:34.649309 15886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:34.650171 15878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:34.651276 15878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:34.652390 15878 hybrid_clock.cc:648] HybridClock initialized: now 1785990874652369 us; error 30 us; skew 500 ppm
W20260806 04:34:35.439898 15878 net_util.cc:504] Time spent look up canonical hostname for localhost 'dist-test-slave-xt4k': real 0.787s user 0.001s sys 0.000s
Aug 06 04:34:35 dist-test-slave-xt4k krb5kdc[15857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990875, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:35.442106 15878 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:35.443158 15878 webserver.cc:460] Webserver started at http://127.12.15.254:44763/ using document root <none> and password file <none>
I20260806 04:34:35.443738 15878 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:35.443807 15878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:35.444017 15878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:35.446210 15878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "8d60535cb3064ab7a47291eaef59e808"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "82a546b22bbe7500538925dea5b5d253"
server_key_iv: "7d8314a57208255b76e7ac3243f10093"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.446784 15878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "8d60535cb3064ab7a47291eaef59e808"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "82a546b22bbe7500538925dea5b5d253"
server_key_iv: "7d8314a57208255b76e7ac3243f10093"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.450484 15878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:34:35.452955 15900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.454202 15878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:35.454322 15878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "8d60535cb3064ab7a47291eaef59e808"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "82a546b22bbe7500538925dea5b5d253"
server_key_iv: "7d8314a57208255b76e7ac3243f10093"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.454417 15878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:35.466734 15878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:35.469619 15878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:35.469789 15878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:35.477468 15878 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36665
I20260806 04:34:35.477530 15983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36665 every 8 connection(s)
I20260806 04:34:35.478502 15878 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:35.481513 15984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:35.486531 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 15878
I20260806 04:34:35.486660 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:35.486919 12351 external_mini_cluster.cc:1428] Setting key a88f6c9801945f2a79a30ff48f9ff879
I20260806 04:34:35.487490 15984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808: Bootstrap starting.
I20260806 04:34:35.489977 15984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:35.490947 15984 log.cc:824] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:35.493083 15984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808: No bootstrap required, opened a new log
Aug 06 04:34:35 dist-test-slave-xt4k krb5kdc[15857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:35.497478 15984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d60535cb3064ab7a47291eaef59e808" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36665 } }
I20260806 04:34:35.497754 15984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:35.497828 15984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d60535cb3064ab7a47291eaef59e808, State: Initialized, Role: FOLLOWER
I20260806 04:34:35.498260 15984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d60535cb3064ab7a47291eaef59e808" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36665 } }
I20260806 04:34:35.498389 15984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:35.498471 15984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:35.498564 15984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:35.499455 15984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d60535cb3064ab7a47291eaef59e808" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36665 } }
I20260806 04:34:35.499881 15984 leader_election.cc:302] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d60535cb3064ab7a47291eaef59e808; no voters:
I20260806 04:34:35.500226 15984 leader_election.cc:288] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:35.500325 15989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:35.500551 15989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [term 1 LEADER]: Becoming Leader. State: Replica: 8d60535cb3064ab7a47291eaef59e808, State: Running, Role: LEADER
I20260806 04:34:35.500895 15989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d60535cb3064ab7a47291eaef59e808" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36665 } }
I20260806 04:34:35.501586 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:35.488358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.12.15.254:36665)
0806 04:34:35.488772 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:35.488788 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:35.488824 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:34:35.489454 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:35.489470 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:35.489816 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:35.490115 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:35.491014 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:35.492007 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:35.493152 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:35.493636 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:35.493756 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:35.496571 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:35.496576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:35.496591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:35.496619 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:35.498321 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:35.498787 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:35.498794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:35.498799 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:35.498925 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:35.499200 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:35.499204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:35.499205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:35.499670 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:35.499850 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:35.500137 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 04:34:35.500649 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
I20260806 04:34:35.501653 15984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:35.502873 15993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d60535cb3064ab7a47291eaef59e808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d60535cb3064ab7a47291eaef59e808" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36665 } } }
I20260806 04:34:35.502914 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d60535cb3064ab7a47291eaef59e808. Latest consensus state: current_term: 1 leader_uuid: "8d60535cb3064ab7a47291eaef59e808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d60535cb3064ab7a47291eaef59e808" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36665 } } }
I20260806 04:34:35.503012 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:35.503010 15993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:35.504560 15997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:35.507649 15997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:35.513710 15997 catalog_manager.cc:1347] Generated new cluster ID: d5fa5483ef494c1eae534ef4026e9457
I20260806 04:34:35.513780 15997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:35.547931 15997 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:35.548938 15997 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:35.552771 15997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d60535cb3064ab7a47291eaef59e808: Generated new TSK 0
I20260806 04:34:35.553447 15997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:35.586634 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36665
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:35.703671 16020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:35.703922 16020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:35.703962 16020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:35.708019 16020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:35.708073 16020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:35.708129 16020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:35.713359 16020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:35.714296 16020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:35.715258 16020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:34:35.725071 16020 server_base.cc:1034] running on GCE node
W20260806 04:34:35.724978 16026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:35.725015 16027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:35.725025 16029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:35.725654 16020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:35.726286 16020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:35.727459 16020 hybrid_clock.cc:648] HybridClock initialized: now 1785990875727444 us; error 24 us; skew 500 ppm
Aug 06 04:34:35 dist-test-slave-xt4k krb5kdc[15857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990875, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:35.730296 16020 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:35.731161 16020 webserver.cc:460] Webserver started at http://127.12.15.193:35619/ using document root <none> and password file <none>
I20260806 04:34:35.731694 16020 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:35.731761 16020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:35.731971 16020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:35.734026 16020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "551c9d53f3fe46338ba3a906413e4f78"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "5ee0bba1adec8baa611cf4f02e444031"
server_key_iv: "baa2c3432df9735d171a6707c5d1621e"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.734558 16020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "551c9d53f3fe46338ba3a906413e4f78"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "5ee0bba1adec8baa611cf4f02e444031"
server_key_iv: "baa2c3432df9735d171a6707c5d1621e"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.738190 16020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 04:34:35.740561 16042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.741775 16020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:35.741887 16020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "551c9d53f3fe46338ba3a906413e4f78"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "5ee0bba1adec8baa611cf4f02e444031"
server_key_iv: "baa2c3432df9735d171a6707c5d1621e"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.741981 16020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:35.771418 16020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:35.774504 16020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:35.774681 16020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:35.775274 16020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:35.776093 16020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:35.776141 16020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.776196 16020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:35.776223 16020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.784441 16020 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:42699
I20260806 04:34:35.784477 16222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:42699 every 8 connection(s)
I20260806 04:34:35.785284 16020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
Aug 06 04:34:35 dist-test-slave-xt4k krb5kdc[15857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990875, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:35.794433 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 16020
I20260806 04:34:35.794555 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:35.794874 12351 external_mini_cluster.cc:1428] Setting key 74ca918b87c6a1804b36deda046e6a1b
I20260806 04:34:35.797866 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:35.787001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:45929 (local address 127.12.15.254:36665)
0806 04:34:35.787158 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:35.787163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:35.788178 (+ 1015us) server_negotiation.cc:407] Connection header received
0806 04:34:35.789016 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:35.789019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:35.789071 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:35.789143 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:35.790219 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:35.790745 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:35.791887 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:35.792127 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:35.792220 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:35.794518 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:35.794523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:35.794526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:35.794554 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:35.795866 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:35.796438 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:35.796442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:35.796444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:35.796523 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:35.796837 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:35.796840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:35.796841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:35.797021 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:35.797106 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:35.797583 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 04:34:35.797706 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:34:35.798691 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:35.787349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36665 (local address 127.12.15.193:45929)
0806 04:34:35.788038 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:35.788079 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:35.788825 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:35.789297 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:35.789303 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:35.789637 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:35.790105 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:35.790119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:35.790830 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:35.790832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:35.791683 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:35.791692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:35.792266 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:35.792269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:35.792435 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:35.792959 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:35.792974 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:35.794334 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:35.795982 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:35.795988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:35.796002 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:35.796331 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:35.796605 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:35.796608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:35.796610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:35.796742 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:35.797118 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:35.797125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:35.797354 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 04:34:35.797516 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 04:34:35.797824 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":119,"threads_started":1}
I20260806 04:34:35.799899 16223 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36665
I20260806 04:34:35.800122 16223 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:35.800626 16223 heartbeater.cc:511] Master 127.12.15.254:36665 requested a full tablet report, sending...
I20260806 04:34:35.802371 15925 ts_manager.cc:194] Registered new tserver with Master: 551c9d53f3fe46338ba3a906413e4f78 (127.12.15.193:42699)
I20260806 04:34:35.803957 15925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:45929
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:35.821426 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36665
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:35.932374 16229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:35.932659 16229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:35.932705 16229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:35.937256 16229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:35.937331 16229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:35.937397 16229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:35.942922 16229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:35.944021 16229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:35.945107 16229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:35.954959 16237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:35.955046 16241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:35.954959 16238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:35.954998 16229 server_base.cc:1034] running on GCE node
I20260806 04:34:35.955607 16229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:35.956250 16229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:35.957418 16229 hybrid_clock.cc:648] HybridClock initialized: now 1785990875957376 us; error 53 us; skew 500 ppm
Aug 06 04:34:35 dist-test-slave-xt4k krb5kdc[15857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990875, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:35.960011 16229 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:35.960898 16229 webserver.cc:460] Webserver started at http://127.12.15.194:38035/ using document root <none> and password file <none>
I20260806 04:34:35.961443 16229 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:35.961510 16229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:35.961714 16229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:35.963709 16229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "80282212bc85418582a9bbe95b9d7ce4"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "29f75250199f35be27c8eb332afbf283"
server_key_iv: "e4ea32003e41ae5af6557243c1225530"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.964262 16229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "80282212bc85418582a9bbe95b9d7ce4"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "29f75250199f35be27c8eb332afbf283"
server_key_iv: "e4ea32003e41ae5af6557243c1225530"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.967952 16229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:35.970376 16250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.971611 16229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:35.971721 16229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "80282212bc85418582a9bbe95b9d7ce4"
format_stamp: "Formatted at 2026-08-06 04:34:35 on dist-test-slave-xt4k"
server_key: "29f75250199f35be27c8eb332afbf283"
server_key_iv: "e4ea32003e41ae5af6557243c1225530"
server_key_version: "encryptionkey@0"
I20260806 04:34:35.971812 16229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:35.984603 16229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:35.987511 16229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:35.987686 16229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:35.988317 16229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:35.989171 16229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:35.989217 16229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.989271 16229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:35.989298 16229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:35.997605 16229 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:33649
I20260806 04:34:35.997648 16424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:33649 every 8 connection(s)
I20260806 04:34:35.998483 16229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:36.007524 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 16229
I20260806 04:34:36.007632 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
Aug 06 04:34:36 dist-test-slave-xt4k krb5kdc[15857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990875, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:36.007917 12351 external_mini_cluster.cc:1428] Setting key 03dd787a33b51f940de2c11900d1d8a9
I20260806 04:34:36.012625 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:36.000190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:52919 (local address 127.12.15.254:36665)
0806 04:34:36.000314 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:36.000318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:36.001525 (+ 1207us) server_negotiation.cc:407] Connection header received
0806 04:34:36.002420 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:36.002424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:36.002474 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:36.002581 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:36.003749 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.004294 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.005477 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.005703 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.005790 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:36.008581 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:36.008584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:36.008587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:36.008611 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:36.010240 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:36.010867 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:36.010870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:36.010871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:36.010961 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:36.011349 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:36.011353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:36.011354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:36.011575 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:36.011656 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:36.012323 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 04:34:36.012488 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:34:36.013701 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:36.000631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36665 (local address 127.12.15.194:52919)
0806 04:34:36.001326 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:36.001366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:36.002189 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:36.002704 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:36.002711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:36.003076 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:36.003612 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:36.003627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.004426 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.004429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:36.005378 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:36.005388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.005864 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.005867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:36.006056 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:36.006626 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:36.006642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:36.008379 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:36.010364 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:36.010372 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:36.010390 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:36.010746 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:36.011050 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:36.011053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:36.011054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:36.011240 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:36.011671 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:36.011678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:36.012029 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 04:34:36.012249 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 04:34:36.012645 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":107,"threads_started":1}
I20260806 04:34:36.014899 16425 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36665
I20260806 04:34:36.015177 16425 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:36.015798 16425 heartbeater.cc:511] Master 127.12.15.254:36665 requested a full tablet report, sending...
I20260806 04:34:36.016999 15925 ts_manager.cc:194] Registered new tserver with Master: 80282212bc85418582a9bbe95b9d7ce4 (127.12.15.194:33649)
I20260806 04:34:36.017590 15925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:52919
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:36.037232 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36665
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:36.149530 16432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:36.149793 16432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:36.149835 16432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:36.153792 16432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:36.153844 16432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:36.153901 16432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:36.159024 16432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:36.159970 16432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:36.160934 16432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:36.170219 16437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:36.170387 16442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:36.170210 16438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:36.170898 16432 server_base.cc:1034] running on GCE node
I20260806 04:34:36.171324 16432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:36.171969 16432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:36.173142 16432 hybrid_clock.cc:648] HybridClock initialized: now 1785990876173131 us; error 26 us; skew 500 ppm
Aug 06 04:34:36 dist-test-slave-xt4k krb5kdc[15857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990876, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:36.175879 16432 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:36.176761 16432 webserver.cc:460] Webserver started at http://127.12.15.195:34335/ using document root <none> and password file <none>
I20260806 04:34:36.177284 16432 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:36.177345 16432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:36.177539 16432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:36.179497 16432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "78d5fa0579734e509f8c50ce0a2dd996"
format_stamp: "Formatted at 2026-08-06 04:34:36 on dist-test-slave-xt4k"
server_key: "b3a64392b8a9b8e32fc6502ec1b7047b"
server_key_iv: "26240711b158a48b4106b084f10eec3b"
server_key_version: "encryptionkey@0"
I20260806 04:34:36.180030 16432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "78d5fa0579734e509f8c50ce0a2dd996"
format_stamp: "Formatted at 2026-08-06 04:34:36 on dist-test-slave-xt4k"
server_key: "b3a64392b8a9b8e32fc6502ec1b7047b"
server_key_iv: "26240711b158a48b4106b084f10eec3b"
server_key_version: "encryptionkey@0"
I20260806 04:34:36.183831 16432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:34:36.186237 16462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:36.187472 16432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:36.187630 16432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "78d5fa0579734e509f8c50ce0a2dd996"
format_stamp: "Formatted at 2026-08-06 04:34:36 on dist-test-slave-xt4k"
server_key: "b3a64392b8a9b8e32fc6502ec1b7047b"
server_key_iv: "26240711b158a48b4106b084f10eec3b"
server_key_version: "encryptionkey@0"
I20260806 04:34:36.187740 16432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:36.221586 16432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:36.224810 16432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:36.224988 16432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:36.225585 16432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:36.226416 16432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:36.226464 16432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:36.226503 16432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:36.226532 16432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:36.234620 16432 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:44877
I20260806 04:34:36.234663 16635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:44877 every 8 connection(s)
I20260806 04:34:36.235472 16432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:36.244191 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 16432
Aug 06 04:34:36 dist-test-slave-xt4k krb5kdc[15857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990876, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:36.244306 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:36.244681 12351 external_mini_cluster.cc:1428] Setting key 998c69b8928392c905ec7a04eb9d2e51
I20260806 04:34:36.248315 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:36.237292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:36425 (local address 127.12.15.254:36665)
0806 04:34:36.237454 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:36.237459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:36.238481 (+ 1022us) server_negotiation.cc:407] Connection header received
0806 04:34:36.239330 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:36.239333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:36.239380 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:36.239442 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:36.240645 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.241176 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.242271 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.242502 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.242593 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:36.244923 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:36.244927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:36.244929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:36.244956 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:36.246327 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:36.246899 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:36.246903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:36.246905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:36.246998 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:36.247308 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:36.247311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:36.247313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:36.247490 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:36.247576 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:36.248044 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 04:34:36.248159 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:34:36.249078 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:36.237679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36665 (local address 127.12.15.195:36425)
0806 04:34:36.238343 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:36.238383 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:36.239147 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:36.239617 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:36.239623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:36.239957 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:36.240490 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:36.240522 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.241276 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.241279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:36.242146 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:36.242155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:36.242630 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:36.242633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:36.242770 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:36.243240 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:36.243256 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:36.244748 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:36.246459 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:36.246465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:36.246479 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:36.246807 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:36.247079 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:36.247083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:36.247085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:36.247223 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:36.247589 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:36.247595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:36.247819 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 04:34:36.247979 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 04:34:36.248280 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":110,"threads_started":1}
I20260806 04:34:36.250241 16636 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36665
I20260806 04:34:36.250458 16636 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:36.250929 16636 heartbeater.cc:511] Master 127.12.15.254:36665 requested a full tablet report, sending...
I20260806 04:34:36.251892 15925 ts_manager.cc:194] Registered new tserver with Master: 78d5fa0579734e509f8c50ce0a2dd996 (127.12.15.195:44877)
I20260806 04:34:36.252467 15925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:36425
I20260806 04:34:36.258350 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 04:34:36.272465 15987 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:36.270963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.12.15.254:36665)
0806 04:34:36.271110 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:36.271115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:36.271270 (+ 155us) server_negotiation.cc:407] Connection header received
0806 04:34:36.271412 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:36.271415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:36.271459 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:36.271528 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:36.272293 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49018: BlockingRecv error: recv got EOF from 127.0.0.1:49018 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:34:36.273485 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 16020
I20260806 04:34:36.283510 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 16229
I20260806 04:34:36.303146 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 16432
I20260806 04:34:36.315160 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 15878
2026-08-06T04:34:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3069 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:36 dist-test-slave-xt4k krb5kdc[16651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:36 dist-test-slave-xt4k krb5kdc[16651](info): set up 2 sockets
Aug 06 04:34:36 dist-test-slave-xt4k krb5kdc[16651](info): commencing operation
krb5kdc: starting...
W20260806 04:34:37.467860 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:37 dist-test-slave-xt4k krb5kdc[16651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:37.591547 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39811
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:39069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:39811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:37.703106 16672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:37.703387 16672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:37.703435 16672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:37.707378 16672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:37.707441 16672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:37.707461 16672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:37.707475 16672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:37.707489 16672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:37.712755 16672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:39069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:39811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:37.713804 16672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:37.714784 16672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:37.724290 16678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:37.724236 16677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:37.724315 16672 server_base.cc:1034] running on GCE node
W20260806 04:34:37.724277 16681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:37.725117 16672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:37.726224 16672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:37.727389 16672 hybrid_clock.cc:648] HybridClock initialized: now 1785990877727373 us; error 30 us; skew 500 ppm
Aug 06 04:34:37 dist-test-slave-xt4k krb5kdc[16651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990877, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:37.730094 16672 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:37.730949 16672 webserver.cc:460] Webserver started at http://127.12.15.254:33975/ using document root <none> and password file <none>
I20260806 04:34:37.731491 16672 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:37.731550 16672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:37.731748 16672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:37.733819 16672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "915c5ffb7f20409382274f686f42b11d"
format_stamp: "Formatted at 2026-08-06 04:34:37 on dist-test-slave-xt4k"
server_key: "020fba491dbbc72687adf470bd394010"
server_key_iv: "9d1b43c7c2bdf1d4c8ee45dd5eb6023e"
server_key_version: "encryptionkey@0"
I20260806 04:34:37.734365 16672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "915c5ffb7f20409382274f686f42b11d"
format_stamp: "Formatted at 2026-08-06 04:34:37 on dist-test-slave-xt4k"
server_key: "020fba491dbbc72687adf470bd394010"
server_key_iv: "9d1b43c7c2bdf1d4c8ee45dd5eb6023e"
server_key_version: "encryptionkey@0"
I20260806 04:34:37.738106 16672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:34:37.740590 16692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:37.741834 16672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:37.741971 16672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "915c5ffb7f20409382274f686f42b11d"
format_stamp: "Formatted at 2026-08-06 04:34:37 on dist-test-slave-xt4k"
server_key: "020fba491dbbc72687adf470bd394010"
server_key_iv: "9d1b43c7c2bdf1d4c8ee45dd5eb6023e"
server_key_version: "encryptionkey@0"
I20260806 04:34:37.742075 16672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:37.758603 16672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:37.761518 16672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:37.761696 16672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:37.768743 16672 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:39811
I20260806 04:34:37.769102 16769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:39811 every 8 connection(s)
I20260806 04:34:37.769726 16672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:37.772619 16772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:37.777817 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 16672
I20260806 04:34:37.777949 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:37.778242 12351 external_mini_cluster.cc:1428] Setting key 282590633791ed0cad87de5a97136a3a
I20260806 04:34:37.778692 16772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d: Bootstrap starting.
I20260806 04:34:37.781148 16772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:37.782050 16772 log.cc:824] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:37.783995 16772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d: No bootstrap required, opened a new log
Aug 06 04:34:37 dist-test-slave-xt4k krb5kdc[16651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:37.788185 16772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "915c5ffb7f20409382274f686f42b11d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39811 } }
I20260806 04:34:37.788473 16772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:37.788534 16772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915c5ffb7f20409382274f686f42b11d, State: Initialized, Role: FOLLOWER
I20260806 04:34:37.788980 16772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "915c5ffb7f20409382274f686f42b11d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39811 } }
I20260806 04:34:37.789105 16772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:37.789163 16772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:37.789258 16772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:37.790316 16772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "915c5ffb7f20409382274f686f42b11d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39811 } }
I20260806 04:34:37.790693 16772 leader_election.cc:302] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 915c5ffb7f20409382274f686f42b11d; no voters:
I20260806 04:34:37.791002 16772 leader_election.cc:288] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:37.791106 16781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:37.791402 16781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [term 1 LEADER]: Becoming Leader. State: Replica: 915c5ffb7f20409382274f686f42b11d, State: Running, Role: LEADER
I20260806 04:34:37.791879 16781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "915c5ffb7f20409382274f686f42b11d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39811 } }
I20260806 04:34:37.792434 16772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:37.793035 16783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 915c5ffb7f20409382274f686f42b11d. Latest consensus state: current_term: 1 leader_uuid: "915c5ffb7f20409382274f686f42b11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "915c5ffb7f20409382274f686f42b11d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39811 } } }
I20260806 04:34:37.793169 16783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:37.793172 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:37.779547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.12.15.254:39811)
0806 04:34:37.779964 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:37.779980 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:37.780018 (+ 38us) server_negotiation.cc:407] Connection header received
0806 04:34:37.780745 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:37.780766 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:37.781103 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:37.781489 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:37.782372 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:37.783362 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:37.784489 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:37.784964 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:37.785083 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:37.787818 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:37.787823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:37.787836 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:37.787864 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:37.789659 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:37.790178 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:37.790182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:37.790187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:37.790311 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:37.790590 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:37.790592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:37.790593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:37.791056 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:37.791308 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:37.791888 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 04:34:37.792194 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":118,"threads_started":1}
I20260806 04:34:37.793534 16785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "915c5ffb7f20409382274f686f42b11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "915c5ffb7f20409382274f686f42b11d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39811 } } }
I20260806 04:34:37.793629 16785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:37.796478 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 04:34:37.797435 16805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:34:37.797506 16805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:34:37.797581 16790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:37.798363 16790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:37.803229 16790 catalog_manager.cc:1347] Generated new cluster ID: 57f69ffe0972428ea9288b83b46ab76e
I20260806 04:34:37.803323 16790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:37.811069 16790 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:37.811987 16790 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:37.817526 16790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 915c5ffb7f20409382274f686f42b11d: Generated new TSK 0
I20260806 04:34:37.818198 16790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:34:37.820183 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:37.814099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.12.15.254:39811)
0806 04:34:37.814205 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:37.814210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:37.814490 (+ 280us) server_negotiation.cc:407] Connection header received
0806 04:34:37.814603 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:37.814606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:37.814660 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:37.814747 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:37.815310 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:37.816127 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:37.817414 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:37.818269 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:37.818347 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:37.819346 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:37.819349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:37.819351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:37.819791 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:37.819822 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:37.819838 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:37.819913 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:37.820022 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 04:34:37.820079 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 04:34:37.824674 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 16672
2026-08-06T04:34:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1518 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:37 dist-test-slave-xt4k krb5kdc[16826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:37 dist-test-slave-xt4k krb5kdc[16826](info): set up 2 sockets
Aug 06 04:34:37 dist-test-slave-xt4k krb5kdc[16826](info): commencing operation
krb5kdc: starting...
W20260806 04:34:38.992677 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:39.119704 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39695
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:39695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:39.235774 16850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:39.236022 16850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:39.236064 16850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:39.240001 16850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:39.240062 16850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:39.240080 16850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:39.240095 16850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:39.240109 16850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:39.245380 16850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:39695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:39.246428 16850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:39.247416 16850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:39.255722 16860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:39.255725 16857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:39.255882 16850 server_base.cc:1034] running on GCE node
W20260806 04:34:39.256148 16856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:39.256759 16850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:39.257844 16850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:39.259025 16850 hybrid_clock.cc:648] HybridClock initialized: now 1785990879259006 us; error 32 us; skew 500 ppm
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:39.261788 16850 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:39.262678 16850 webserver.cc:460] Webserver started at http://127.12.15.254:33473/ using document root <none> and password file <none>
I20260806 04:34:39.263257 16850 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:39.263324 16850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:39.263607 16850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:39.265710 16850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "d7c046264584410a8f88377a410d686d"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "c61861058b5da9472ef3e38508fda8ae"
server_key_iv: "57f36715b2b37ce415f8dcde470d4ac6"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.266253 16850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "d7c046264584410a8f88377a410d686d"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "c61861058b5da9472ef3e38508fda8ae"
server_key_iv: "57f36715b2b37ce415f8dcde470d4ac6"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.270123 16850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:39.272629 16872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.274058 16850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:34:39.274186 16850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "d7c046264584410a8f88377a410d686d"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "c61861058b5da9472ef3e38508fda8ae"
server_key_iv: "57f36715b2b37ce415f8dcde470d4ac6"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.274291 16850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:39.289961 16850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:39.292838 16850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:39.293010 16850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:39.299691 16850 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:39695
I20260806 04:34:39.299957 16953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:39695 every 8 connection(s)
I20260806 04:34:39.300697 16850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:39.303494 16956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:39.306815 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 16850
I20260806 04:34:39.306917 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:39.307180 12351 external_mini_cluster.cc:1428] Setting key ec324b2fa177836d04d9c9af22d78284
I20260806 04:34:39.309340 16956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d: Bootstrap starting.
I20260806 04:34:39.312072 16956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:39.312986 16956 log.cc:824] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:39.315145 16956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d: No bootstrap required, opened a new log
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:39.318634 16956 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7c046264584410a8f88377a410d686d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39695 } }
I20260806 04:34:39.318910 16956 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:39.318958 16956 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c046264584410a8f88377a410d686d, State: Initialized, Role: FOLLOWER
I20260806 04:34:39.319871 16956 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7c046264584410a8f88377a410d686d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39695 } }
I20260806 04:34:39.320079 16956 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:39.320155 16956 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:39.320281 16956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:39.321290 16956 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7c046264584410a8f88377a410d686d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39695 } }
I20260806 04:34:39.321719 16956 leader_election.cc:302] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7c046264584410a8f88377a410d686d; no voters:
I20260806 04:34:39.322113 16956 leader_election.cc:288] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:39.322180 16962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:39.322422 16962 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [term 1 LEADER]: Becoming Leader. State: Replica: d7c046264584410a8f88377a410d686d, State: Running, Role: LEADER
I20260806 04:34:39.322777 16962 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7c046264584410a8f88377a410d686d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39695 } }
I20260806 04:34:39.322831 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:39.308602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.12.15.254:39695)
0806 04:34:39.309056 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:39.309074 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:39.309117 (+ 43us) server_negotiation.cc:407] Connection header received
0806 04:34:39.309917 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:39.309942 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:39.310348 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:39.310731 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:39.311408 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.312937 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.313853 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.314497 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.314664 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:39.317341 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:39.317346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:39.317359 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:39.317386 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:39.319695 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:39.320275 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:39.320281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:39.320287 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:39.320480 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:39.320817 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:39.320820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:39.320821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:39.321240 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:39.321420 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:39.321694 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 04:34:39.321957 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":128,"threads_started":1}
I20260806 04:34:39.323462 16956 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:39.325019 16965 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7c046264584410a8f88377a410d686d. Latest consensus state: current_term: 1 leader_uuid: "d7c046264584410a8f88377a410d686d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7c046264584410a8f88377a410d686d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39695 } } }
I20260806 04:34:39.325148 16965 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:39.325441 16963 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7c046264584410a8f88377a410d686d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7c046264584410a8f88377a410d686d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39695 } } }
I20260806 04:34:39.325544 16963 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:39.325897 16986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:39.328819 16986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:39.334221 16986 catalog_manager.cc:1347] Generated new cluster ID: 601c789cc2b349a39827629332e060a6
I20260806 04:34:39.334285 16986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:39.351588 16986 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:39.352571 16986 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:39.358724 16986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7c046264584410a8f88377a410d686d: Generated new TSK 0
I20260806 04:34:39.359404 16986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:39.399310 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39695
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:39.519698 17001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:39.519966 17001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:39.520010 17001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:39.524384 17001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:39.524454 17001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:39.524514 17001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:39.529902 17001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:39.530885 17001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:39.531843 17001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:39.541492 17010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:39.541497 17013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:39.541651 17001 server_base.cc:1034] running on GCE node
W20260806 04:34:39.541831 17011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:39.542204 17001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:39.542868 17001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:39.544018 17001 hybrid_clock.cc:648] HybridClock initialized: now 1785990879543996 us; error 35 us; skew 500 ppm
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:39.546933 17001 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:39.547750 17001 webserver.cc:460] Webserver started at http://127.12.15.193:34081/ using document root <none> and password file <none>
I20260806 04:34:39.548264 17001 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:39.548319 17001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:39.548537 17001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:39.550475 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "1475235a50054b9d8c8e465986726c72"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "f5b0074730433c56b1661be3cc01f858"
server_key_iv: "90fd19d589568f1348dc059b6f7450ea"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.551038 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "1475235a50054b9d8c8e465986726c72"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "f5b0074730433c56b1661be3cc01f858"
server_key_iv: "90fd19d589568f1348dc059b6f7450ea"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.554533 17001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 04:34:39.556881 17025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.558066 17001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:39.558183 17001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "1475235a50054b9d8c8e465986726c72"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "f5b0074730433c56b1661be3cc01f858"
server_key_iv: "90fd19d589568f1348dc059b6f7450ea"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.558281 17001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:39.598685 17001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:39.601513 17001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:39.601686 17001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:39.602279 17001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:39.603121 17001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:39.603178 17001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.603231 17001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:39.603259 17001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.611696 17001 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:34395
I20260806 04:34:39.611749 17213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:34395 every 8 connection(s)
I20260806 04:34:39.612632 17001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:39.616997 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 17001
I20260806 04:34:39.617138 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:39.617449 12351 external_mini_cluster.cc:1428] Setting key df9a2d6d1a69167c9b4c31c9e62bd272
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:39.626962 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:39.614447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:42583 (local address 127.12.15.254:39695)
0806 04:34:39.614599 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:39.614604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:39.615398 (+ 794us) server_negotiation.cc:407] Connection header received
0806 04:34:39.616754 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:39.616756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:39.616805 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:39.616875 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:39.618330 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.619070 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.620582 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.620823 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.620948 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:39.623497 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:39.623500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:39.623502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:39.623525 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:39.624972 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:39.625506 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:39.625510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:39.625511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:39.625592 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:39.625909 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:39.625912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:39.625914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:39.626096 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:39.626190 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:39.626662 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 04:34:39.626794 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:39.627725 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:39.614804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39695 (local address 127.12.15.193:42583)
0806 04:34:39.615250 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:39.615291 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:39.616531 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:39.617078 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:39.617088 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:39.617563 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:39.618177 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:39.618196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.619198 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.619201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:39.620434 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:39.620448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.620992 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.620996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:39.621166 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:39.621730 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:39.621747 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:39.623325 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:39.625140 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:39.625148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:39.625163 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:39.625409 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:39.625683 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:39.625686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:39.625688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:39.625823 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:39.626200 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:39.626207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:39.626436 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 04:34:39.626609 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 04:34:39.626922 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260806 04:34:39.628901 17214 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39695
I20260806 04:34:39.629120 17214 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:39.629578 17214 heartbeater.cc:511] Master 127.12.15.254:39695 requested a full tablet report, sending...
I20260806 04:34:39.631121 16894 ts_manager.cc:194] Registered new tserver with Master: 1475235a50054b9d8c8e465986726c72 (127.12.15.193:34395)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:34:39.632315 16894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:42583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:39.646785 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39695
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:39.759229 17222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:39.759464 17222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:39.759503 17222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:39.763612 17222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:39.763667 17222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:39.763733 17222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:39.768849 17222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:39.769887 17222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:39.770876 17222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:39.780293 17234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:39.780303 17229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:39.780493 17222 server_base.cc:1034] running on GCE node
W20260806 04:34:39.780387 17232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:39.781000 17222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:39.781601 17222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:39.782745 17222 hybrid_clock.cc:648] HybridClock initialized: now 1785990879782718 us; error 37 us; skew 500 ppm
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:39.785405 17222 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:39.786247 17222 webserver.cc:460] Webserver started at http://127.12.15.194:37069/ using document root <none> and password file <none>
I20260806 04:34:39.786765 17222 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:39.786821 17222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:39.786986 17222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:39.789003 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "a2be14c69b394140b7a9890dcdd3cd1e"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "edf896785c48770e4d243bc194c3bc3b"
server_key_iv: "0554b9084647485e36be00fcb38df939"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.789525 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "a2be14c69b394140b7a9890dcdd3cd1e"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "edf896785c48770e4d243bc194c3bc3b"
server_key_iv: "0554b9084647485e36be00fcb38df939"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.793088 17222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:34:39.795617 17244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.796945 17222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:34:39.797051 17222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "a2be14c69b394140b7a9890dcdd3cd1e"
format_stamp: "Formatted at 2026-08-06 04:34:39 on dist-test-slave-xt4k"
server_key: "edf896785c48770e4d243bc194c3bc3b"
server_key_iv: "0554b9084647485e36be00fcb38df939"
server_key_version: "encryptionkey@0"
I20260806 04:34:39.797164 17222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:39.824798 17222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:39.828002 17222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:39.828194 17222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:39.828845 17222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:39.829676 17222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:39.829725 17222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.829766 17222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:39.829782 17222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:39.839226 17222 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:36793
I20260806 04:34:39.839242 17418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:36793 every 8 connection(s)
I20260806 04:34:39.840189 17222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:39.843585 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 17222
I20260806 04:34:39.843701 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:39.843966 12351 external_mini_cluster.cc:1428] Setting key c7d2bc5276625d24670e11ebbee99611
Aug 06 04:34:39 dist-test-slave-xt4k krb5kdc[16826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990879, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:39.856101 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:39.842120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33523 (local address 127.12.15.254:39695)
0806 04:34:39.842258 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:39.842262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:39.843118 (+ 856us) server_negotiation.cc:407] Connection header received
0806 04:34:39.844662 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:39.844666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:39.844730 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:39.844814 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:39.846437 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.847131 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.848659 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.848986 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.849094 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:39.852149 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:39.852154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:39.852158 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:39.852192 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:39.853880 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:39.854459 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:39.854463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:39.854465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:39.854564 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:39.854929 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:39.854932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:39.854934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:39.855143 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:39.855237 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:39.855790 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 04:34:39.855926 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:34:39.857054 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:39.842463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39695 (local address 127.12.15.194:33523)
0806 04:34:39.842962 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:39.843007 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:39.844417 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:39.845021 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:39.845031 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:39.845544 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:39.846254 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:39.846277 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.847279 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.847283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:39.848526 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:39.848538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:39.849165 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:39.849293 (+ 128us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:39.849498 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:39.850125 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:39.850143 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:39.851935 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:39.854024 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:39.854030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:39.854045 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:39.854339 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:39.854652 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:39.854655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:39.854656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:39.854816 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:39.855253 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:39.855261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:39.855525 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 04:34:39.855719 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 04:34:39.856086 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:34:39.858376 17422 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39695
I20260806 04:34:39.858688 17422 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:39.859303 17422 heartbeater.cc:511] Master 127.12.15.254:39695 requested a full tablet report, sending...
I20260806 04:34:39.860643 16894 ts_manager.cc:194] Registered new tserver with Master: a2be14c69b394140b7a9890dcdd3cd1e (127.12.15.194:36793)
I20260806 04:34:39.861456 16894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:33523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:39.875761 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39695
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:39.989996 17428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:39.990226 17428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:39.990265 17428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:39.994194 17428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:39.994248 17428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:39.994306 17428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:39.999402 17428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:40.000409 17428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:40.001425 17428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:40.011010 17436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:40.010998 17428 server_base.cc:1034] running on GCE node
W20260806 04:34:40.011022 17444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:40.010991 17435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:40.011710 17428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:40.012355 17428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:40.013573 17428 hybrid_clock.cc:648] HybridClock initialized: now 1785990880013540 us; error 33 us; skew 500 ppm
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990880, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:40.016374 17428 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:40.017306 17428 webserver.cc:460] Webserver started at http://127.12.15.195:33405/ using document root <none> and password file <none>
I20260806 04:34:40.017864 17428 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:40.017930 17428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:40.018137 17428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:40.020187 17428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "c9da251c9bba43dba0c51f646a34ffd3"
format_stamp: "Formatted at 2026-08-06 04:34:40 on dist-test-slave-xt4k"
server_key: "4e729def7f99f52f6657d2951d110f68"
server_key_iv: "5aae8317fd467c21d18885d98763deb0"
server_key_version: "encryptionkey@0"
I20260806 04:34:40.020778 17428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "c9da251c9bba43dba0c51f646a34ffd3"
format_stamp: "Formatted at 2026-08-06 04:34:40 on dist-test-slave-xt4k"
server_key: "4e729def7f99f52f6657d2951d110f68"
server_key_iv: "5aae8317fd467c21d18885d98763deb0"
server_key_version: "encryptionkey@0"
I20260806 04:34:40.024638 17428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:34:40.027189 17455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:40.028488 17428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:40.028618 17428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "c9da251c9bba43dba0c51f646a34ffd3"
format_stamp: "Formatted at 2026-08-06 04:34:40 on dist-test-slave-xt4k"
server_key: "4e729def7f99f52f6657d2951d110f68"
server_key_iv: "5aae8317fd467c21d18885d98763deb0"
server_key_version: "encryptionkey@0"
I20260806 04:34:40.028714 17428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:40.045024 17428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:40.048065 17428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:40.048241 17428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:40.048929 17428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:40.049762 17428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:40.049808 17428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:40.049862 17428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:40.049889 17428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:40.058316 17428 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:42281
I20260806 04:34:40.058344 17630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:42281 every 8 connection(s)
I20260806 04:34:40.059157 17428 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:40.063105 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 17428
I20260806 04:34:40.063217 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:40.063512 12351 external_mini_cluster.cc:1428] Setting key 6458b7c555b3df054c7df8bf373b2542
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[16826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990880, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:40.072378 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:40.060995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:34085 (local address 127.12.15.254:39695)
0806 04:34:40.061136 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:40.061140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:40.061930 (+ 790us) server_negotiation.cc:407] Connection header received
0806 04:34:40.063355 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:40.063359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:40.063420 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:40.063497 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:40.064842 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:40.065379 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:40.066487 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:40.066707 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:40.066784 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:40.069078 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:40.069081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:40.069083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:40.069109 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:40.070356 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:40.070895 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:40.070899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:40.070900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:40.071004 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:40.071379 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:40.071382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:40.071384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:40.071585 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:40.071665 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:40.072125 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 04:34:40.072218 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:34:40.073278 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:40.061342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39695 (local address 127.12.15.195:34085)
0806 04:34:40.061792 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:40.061833 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:40.063105 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:40.063694 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:40.063701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:40.064127 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:40.064710 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:40.064731 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:40.065517 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:40.065520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:40.066381 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:40.066391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:40.066818 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:40.066821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:40.066955 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:40.067432 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:40.067475 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:40.068924 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:40.070501 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:40.070506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:40.070518 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:40.070787 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:40.071094 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:40.071096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:40.071098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:40.071260 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:40.071689 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:40.071694 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:40.071932 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 04:34:40.072092 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 04:34:40.072387 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":115,"threads_started":1}
I20260806 04:34:40.074440 17631 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39695
I20260806 04:34:40.074666 17631 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:40.075133 17631 heartbeater.cc:511] Master 127.12.15.254:39695 requested a full tablet report, sending...
I20260806 04:34:40.076161 16894 ts_manager.cc:194] Registered new tserver with Master: c9da251c9bba43dba0c51f646a34ffd3 (127.12.15.195:42281)
I20260806 04:34:40.076884 16894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:34085
I20260806 04:34:40.077289 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990880, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[16826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990880, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:40.102710 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:40.094517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.12.15.254:39695)
0806 04:34:40.094638 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:40.094641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:40.094828 (+ 187us) server_negotiation.cc:407] Connection header received
0806 04:34:40.094911 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:40.094914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:40.094963 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:40.095050 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:40.095752 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:40.096325 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:40.097210 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:40.097440 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:40.097529 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:40.099714 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:40.099720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:40.099722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:40.099748 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:40.101224 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:40.101626 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:40.101630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:40.101631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:40.101717 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:40.101946 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:40.101949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:40.101951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:40.102184 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:40.102292 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:40.102459 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 04:34:40.102564 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 04:34:40.103127 16894 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45696
I20260806 04:34:40.104097 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 17001
I20260806 04:34:40.117373 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 17222
I20260806 04:34:40.129909 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 17428
I20260806 04:34:40.142000 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 16850
2026-08-06T04:34:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2308 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[17654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[17654](info): set up 2 sockets
Aug 06 04:34:40 dist-test-slave-xt4k krb5kdc[17654](info): commencing operation
krb5kdc: starting...
W20260806 04:34:41.297530 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:41 dist-test-slave-xt4k krb5kdc[17654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:41.421689 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:41893
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:41893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:41.533519 17678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:41.533823 17678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:41.533885 17678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:41.537983 17678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:41.538041 17678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:41.538069 17678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:41.538091 17678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:41.538120 17678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:41.543412 17678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:41893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:41893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:41.544499 17678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:41.545498 17678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:41.553646 17685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:41.553617 17687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:41.553822 17678 server_base.cc:1034] running on GCE node
W20260806 04:34:41.553634 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:41.554504 17678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:41.555529 17678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:41.556680 17678 hybrid_clock.cc:648] HybridClock initialized: now 1785990881556655 us; error 37 us; skew 500 ppm
Aug 06 04:34:41 dist-test-slave-xt4k krb5kdc[17654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990881, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:41.562151 17678 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:41.563067 17678 webserver.cc:460] Webserver started at http://127.12.15.254:46811/ using document root <none> and password file <none>
I20260806 04:34:41.563633 17678 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:41.563700 17678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:41.563892 17678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:41.566010 17678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "238f3afe625c43b48bc0a300e5bcde06"
format_stamp: "Formatted at 2026-08-06 04:34:41 on dist-test-slave-xt4k"
server_key: "ad052eb6191dd872f1e4be41c772297b"
server_key_iv: "5743dba7bdb08a3948d122affb940f92"
server_key_version: "encryptionkey@0"
I20260806 04:34:41.566555 17678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "238f3afe625c43b48bc0a300e5bcde06"
format_stamp: "Formatted at 2026-08-06 04:34:41 on dist-test-slave-xt4k"
server_key: "ad052eb6191dd872f1e4be41c772297b"
server_key_iv: "5743dba7bdb08a3948d122affb940f92"
server_key_version: "encryptionkey@0"
I20260806 04:34:41.570303 17678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 04:34:41.572808 17698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:41.574079 17678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:41.574213 17678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "238f3afe625c43b48bc0a300e5bcde06"
format_stamp: "Formatted at 2026-08-06 04:34:41 on dist-test-slave-xt4k"
server_key: "ad052eb6191dd872f1e4be41c772297b"
server_key_iv: "5743dba7bdb08a3948d122affb940f92"
server_key_version: "encryptionkey@0"
I20260806 04:34:41.574311 17678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:41.606426 17678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:41.609483 17678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:41.609691 17678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:41.616912 17678 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:41893
I20260806 04:34:41.617195 17781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:41893 every 8 connection(s)
I20260806 04:34:41.617959 17678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:41.618381 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 17678
I20260806 04:34:41.618523 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:41.618811 12351 external_mini_cluster.cc:1428] Setting key 872f049c3337f258dbce946bed580351
I20260806 04:34:41.621560 17782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 04:34:41 dist-test-slave-xt4k krb5kdc[17654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:41.628149 17782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06: Bootstrap starting.
I20260806 04:34:41.630872 17782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:41.631805 17782 log.cc:824] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:41.633473 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:41.620259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.12.15.254:41893)
0806 04:34:41.620728 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:41.620749 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:41.620795 (+ 46us) server_negotiation.cc:407] Connection header received
0806 04:34:41.621648 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:41.621675 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:41.622093 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:41.622479 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:41.623121 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:41.624627 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:41.625432 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:41.626094 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:41.626250 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:41.628642 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:41.628649 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:41.628666 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:41.628698 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:41.630621 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:41.631070 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:41.631076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:41.631083 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:41.631241 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:41.631472 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:41.631475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:41.631477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:41.632004 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:41.632214 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:41.632460 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 04:34:41.632754 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":122,"threads_started":1}
I20260806 04:34:41.634063 17782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06: No bootstrap required, opened a new log
I20260806 04:34:41.637842 17782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f3afe625c43b48bc0a300e5bcde06" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41893 } }
I20260806 04:34:41.638077 17782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:41.638137 17782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 238f3afe625c43b48bc0a300e5bcde06, State: Initialized, Role: FOLLOWER
I20260806 04:34:41.638537 17782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f3afe625c43b48bc0a300e5bcde06" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41893 } }
I20260806 04:34:41.638674 17782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:41.638739 17782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:41.638836 17782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:41.639734 17782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f3afe625c43b48bc0a300e5bcde06" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41893 } }
I20260806 04:34:41.640049 17782 leader_election.cc:302] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 238f3afe625c43b48bc0a300e5bcde06; no voters:
I20260806 04:34:41.640384 17782 leader_election.cc:288] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:41.640499 17792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:41.640718 17792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [term 1 LEADER]: Becoming Leader. State: Replica: 238f3afe625c43b48bc0a300e5bcde06, State: Running, Role: LEADER
I20260806 04:34:41.641070 17792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f3afe625c43b48bc0a300e5bcde06" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41893 } }
I20260806 04:34:41.641539 17782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:41.642693 17793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "238f3afe625c43b48bc0a300e5bcde06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f3afe625c43b48bc0a300e5bcde06" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41893 } } }
I20260806 04:34:41.642822 17793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:41.642993 17795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 238f3afe625c43b48bc0a300e5bcde06. Latest consensus state: current_term: 1 leader_uuid: "238f3afe625c43b48bc0a300e5bcde06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f3afe625c43b48bc0a300e5bcde06" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41893 } } }
I20260806 04:34:41.643318 17795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:41.643376 17808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:41.646443 17808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:41.651470 17808 catalog_manager.cc:1347] Generated new cluster ID: efb8c133a83749278c08443c7ad1307f
I20260806 04:34:41.651535 17808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:41.677065 17808 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:41.678180 17808 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:41.684572 17808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 238f3afe625c43b48bc0a300e5bcde06: Generated new TSK 0
I20260806 04:34:41.685503 17808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:41.723580 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41893
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:41.837101 17822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:41.837342 17822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:41.837396 17822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:41.841338 17822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:41.841390 17822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:41.841463 17822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:41.846695 17822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:41.847668 17822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:41.848680 17822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:41.858017 17834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:41.858067 17832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:41.858145 17822 server_base.cc:1034] running on GCE node
W20260806 04:34:41.858075 17829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:41.858642 17822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:41.859256 17822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:41.860397 17822 hybrid_clock.cc:648] HybridClock initialized: now 1785990881860376 us; error 29 us; skew 500 ppm
Aug 06 04:34:41 dist-test-slave-xt4k krb5kdc[17654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990881, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:41.863288 17822 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:41.864212 17822 webserver.cc:460] Webserver started at http://127.12.15.193:42697/ using document root <none> and password file <none>
I20260806 04:34:41.864801 17822 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:41.864868 17822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:41.865084 17822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:41.867132 17822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "151a4f43d6e942199538be303ee88ca0"
format_stamp: "Formatted at 2026-08-06 04:34:41 on dist-test-slave-xt4k"
server_key: "37c2baf12b551c5c466e8d5e952c2630"
server_key_iv: "47a4bae7c162b6e670bc2f767c5141a1"
server_key_version: "encryptionkey@0"
I20260806 04:34:41.867762 17822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "151a4f43d6e942199538be303ee88ca0"
format_stamp: "Formatted at 2026-08-06 04:34:41 on dist-test-slave-xt4k"
server_key: "37c2baf12b551c5c466e8d5e952c2630"
server_key_iv: "47a4bae7c162b6e670bc2f767c5141a1"
server_key_version: "encryptionkey@0"
I20260806 04:34:41.871745 17822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 04:34:41.874277 17847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:41.875497 17822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 04:34:41.875630 17822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "151a4f43d6e942199538be303ee88ca0"
format_stamp: "Formatted at 2026-08-06 04:34:41 on dist-test-slave-xt4k"
server_key: "37c2baf12b551c5c466e8d5e952c2630"
server_key_iv: "47a4bae7c162b6e670bc2f767c5141a1"
server_key_version: "encryptionkey@0"
I20260806 04:34:41.875733 17822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:41.897182 17822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:41.900453 17822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:41.900642 17822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:41.901274 17822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:41.902130 17822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:41.902179 17822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:41.902235 17822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:41.902261 17822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:41.910847 17822 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:42219
I20260806 04:34:41.910920 18025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:42219 every 8 connection(s)
I20260806 04:34:41.911755 17822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
Aug 06 04:34:41 dist-test-slave-xt4k krb5kdc[17654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990881, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:41.921247 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 17822
I20260806 04:34:41.921347 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:41.921608 12351 external_mini_cluster.cc:1428] Setting key 1de890db017f36766c44a774bf060c1a
I20260806 04:34:41.924770 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:41.913580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:50919 (local address 127.12.15.254:41893)
0806 04:34:41.913724 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:41.913728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:41.914552 (+ 824us) server_negotiation.cc:407] Connection header received
0806 04:34:41.915878 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:41.915881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:41.915927 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:41.916000 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:41.917141 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:41.917682 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:41.918803 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:41.919011 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:41.919086 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:41.921502 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:41.921505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:41.921507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:41.921531 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:41.922810 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:41.923294 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:41.923297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:41.923298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:41.923371 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:41.923682 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:41.923685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:41.923687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:41.923858 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:41.923943 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:41.924480 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 04:34:41.924609 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:34:41.925613 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:41.913907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41893 (local address 127.12.15.193:50919)
0806 04:34:41.914386 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:41.914428 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:41.915686 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:41.916146 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:41.916153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:41.916551 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:41.917023 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:41.917035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:41.917812 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:41.917816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:41.918705 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:41.918715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:41.919129 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:41.919132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:41.919274 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:41.919844 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:41.919861 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:41.921284 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:41.922931 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:41.922937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:41.922952 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:41.923199 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:41.923452 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:41.923456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:41.923458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:41.923593 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:41.923967 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:41.923973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:41.924216 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 04:34:41.924416 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 04:34:41.924740 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":126,"threads_started":1}
I20260806 04:34:41.926940 18026 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41893
I20260806 04:34:41.927184 18026 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:41.927675 18026 heartbeater.cc:511] Master 127.12.15.254:41893 requested a full tablet report, sending...
I20260806 04:34:41.929271 17723 ts_manager.cc:194] Registered new tserver with Master: 151a4f43d6e942199538be303ee88ca0 (127.12.15.193:42219)
I20260806 04:34:41.930423 17723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:50919
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:41.950778 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41893
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:42.063637 18032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:42.063894 18032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:42.063935 18032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:42.067948 18032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:42.068008 18032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:42.068065 18032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:42.073302 18032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:42.074342 18032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:42.075340 18032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:42.085192 18040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:42.085218 18042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:42.085222 18039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:42.085495 18032 server_base.cc:1034] running on GCE node
I20260806 04:34:42.085943 18032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:42.086584 18032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:42.087762 18032 hybrid_clock.cc:648] HybridClock initialized: now 1785990882087742 us; error 32 us; skew 500 ppm
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:42.090574 18032 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:42.091436 18032 webserver.cc:460] Webserver started at http://127.12.15.194:38643/ using document root <none> and password file <none>
I20260806 04:34:42.091974 18032 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:42.092032 18032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:42.092201 18032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:42.094368 18032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "a737dacff88a41a78babc2e652b10631"
format_stamp: "Formatted at 2026-08-06 04:34:42 on dist-test-slave-xt4k"
server_key: "e0f71478388ededac0d17c637b507282"
server_key_iv: "02bfc10d11eedf4514cd937b1510c0da"
server_key_version: "encryptionkey@0"
I20260806 04:34:42.094928 18032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "a737dacff88a41a78babc2e652b10631"
format_stamp: "Formatted at 2026-08-06 04:34:42 on dist-test-slave-xt4k"
server_key: "e0f71478388ededac0d17c637b507282"
server_key_iv: "02bfc10d11eedf4514cd937b1510c0da"
server_key_version: "encryptionkey@0"
I20260806 04:34:42.099201 18032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 04:34:42.101863 18050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:42.103103 18032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:42.103214 18032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "a737dacff88a41a78babc2e652b10631"
format_stamp: "Formatted at 2026-08-06 04:34:42 on dist-test-slave-xt4k"
server_key: "e0f71478388ededac0d17c637b507282"
server_key_iv: "02bfc10d11eedf4514cd937b1510c0da"
server_key_version: "encryptionkey@0"
I20260806 04:34:42.103327 18032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:42.119372 18032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:42.122334 18032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:42.122527 18032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:42.123116 18032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:42.123929 18032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:42.123976 18032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:42.124022 18032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:42.124051 18032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:42.132437 18032 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:33545
I20260806 04:34:42.132462 18227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:33545 every 8 connection(s)
I20260806 04:34:42.133280 18032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:42.137916 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 18032
I20260806 04:34:42.138011 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:42.138259 12351 external_mini_cluster.cc:1428] Setting key cadd3e5212a4f4f0eafb5649517a58a8
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:42.146962 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:42.134934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:47231 (local address 127.12.15.254:41893)
0806 04:34:42.135100 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:42.135105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:42.135864 (+ 759us) server_negotiation.cc:407] Connection header received
0806 04:34:42.137248 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:42.137252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:42.137304 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:42.137383 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:42.138587 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.139220 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.140382 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.140601 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.140698 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:42.143291 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:42.143296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:42.143299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:42.143325 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:42.144909 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.145461 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.145464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.145465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.145554 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.145907 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.145908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.145910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.146074 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:42.146146 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:42.146738 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 04:34:42.146836 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":57}
I20260806 04:34:42.148083 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:42.135304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41893 (local address 127.12.15.194:47231)
0806 04:34:42.135725 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:42.135765 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:42.137026 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:42.137547 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:42.137554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:42.137909 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:42.138445 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:42.138461 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.139342 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.139345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:42.140250 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:42.140260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.140740 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.140744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:42.140894 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:42.141441 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:42.141464 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:42.143143 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:42.145017 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:42.145022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:42.145032 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:42.145366 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:42.145645 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:42.145648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:42.145650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:42.145824 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:42.146167 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:42.146173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:42.146482 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 04:34:42.146719 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 04:34:42.147058 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:42.149271 18229 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41893
I20260806 04:34:42.149497 18229 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:42.149983 18229 heartbeater.cc:511] Master 127.12.15.254:41893 requested a full tablet report, sending...
I20260806 04:34:42.151055 17723 ts_manager.cc:194] Registered new tserver with Master: a737dacff88a41a78babc2e652b10631 (127.12.15.194:33545)
I20260806 04:34:42.151628 17723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:47231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:42.166827 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41893
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:42.281418 18237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:42.281694 18237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:42.281757 18237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:42.285902 18237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:42.285975 18237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:42.286064 18237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:42.291378 18237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:42.292495 18237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:42.293514 18237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:42.302855 18247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:42.302868 18245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:42.302865 18251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:42.303042 18237 server_base.cc:1034] running on GCE node
I20260806 04:34:42.303529 18237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:42.304157 18237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:42.305327 18237 hybrid_clock.cc:648] HybridClock initialized: now 1785990882305300 us; error 36 us; skew 500 ppm
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:42.308367 18237 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:42.309294 18237 webserver.cc:460] Webserver started at http://127.12.15.195:36593/ using document root <none> and password file <none>
I20260806 04:34:42.309839 18237 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:42.309906 18237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:42.310127 18237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:42.312170 18237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "6ec23647bbfc4a26a30ffe3c5e8b1695"
format_stamp: "Formatted at 2026-08-06 04:34:42 on dist-test-slave-xt4k"
server_key: "6d43abac68333ea41dd2f72d8b73861d"
server_key_iv: "c3b83e7c3a929284da9994477502db8b"
server_key_version: "encryptionkey@0"
I20260806 04:34:42.312806 18237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "6ec23647bbfc4a26a30ffe3c5e8b1695"
format_stamp: "Formatted at 2026-08-06 04:34:42 on dist-test-slave-xt4k"
server_key: "6d43abac68333ea41dd2f72d8b73861d"
server_key_iv: "c3b83e7c3a929284da9994477502db8b"
server_key_version: "encryptionkey@0"
I20260806 04:34:42.316727 18237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:34:42.319201 18260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:42.320598 18237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:42.320725 18237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "6ec23647bbfc4a26a30ffe3c5e8b1695"
format_stamp: "Formatted at 2026-08-06 04:34:42 on dist-test-slave-xt4k"
server_key: "6d43abac68333ea41dd2f72d8b73861d"
server_key_iv: "c3b83e7c3a929284da9994477502db8b"
server_key_version: "encryptionkey@0"
I20260806 04:34:42.320833 18237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:42.335966 18237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:42.339105 18237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:42.339293 18237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:42.339886 18237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:42.340759 18237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:42.340808 18237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:42.340864 18237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:42.340888 18237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:42.349663 18237 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:45501
I20260806 04:34:42.349694 18431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:45501 every 8 connection(s)
I20260806 04:34:42.350538 18237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:42.354089 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 18237
I20260806 04:34:42.354233 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:42.354498 12351 external_mini_cluster.cc:1428] Setting key 476981864219148e37f8dd07a159ac37
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:42.364099 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:42.352283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:53627 (local address 127.12.15.254:41893)
0806 04:34:42.352429 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:42.352432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:42.353303 (+ 871us) server_negotiation.cc:407] Connection header received
0806 04:34:42.354781 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:42.354798 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:42.354870 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:42.354970 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:42.356412 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.357075 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.358204 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.358415 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.358498 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:42.360843 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:42.360846 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:42.360848 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:42.360873 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:42.362106 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.362672 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.362675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.362676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.362758 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.363073 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.363076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.363078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.363264 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:42.363345 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:42.363835 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 04:34:42.363940 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 04:34:42.364886 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:42.352706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41893 (local address 127.12.15.195:53627)
0806 04:34:42.353150 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:42.353198 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:42.354518 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:42.355165 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:42.355174 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:42.355591 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:42.356228 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:42.356248 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.357198 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.357202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:42.358079 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:42.358088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.358532 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.358534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:42.358669 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:42.359234 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:42.359251 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:42.360687 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:42.362212 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:42.362219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:42.362235 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:42.362559 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:42.362828 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:42.362830 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:42.362832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:42.362959 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:42.363342 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:42.363347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:42.363573 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 04:34:42.363724 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 04:34:42.363969 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260806 04:34:42.366096 18432 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41893
I20260806 04:34:42.366315 18432 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:42.366806 18432 heartbeater.cc:511] Master 127.12.15.254:41893 requested a full tablet report, sending...
I20260806 04:34:42.367868 17723 ts_manager.cc:194] Registered new tserver with Master: 6ec23647bbfc4a26a30ffe3c5e8b1695 (127.12.15.195:45501)
I20260806 04:34:42.368412 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:42.368592 17723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:53627
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:42.394891 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:42.385830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.12.15.193:42219)
0806 04:34:42.386103 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:42.386118 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:42.386143 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:34:42.386280 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:42.386287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:42.386484 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:42.386607 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:42.387333 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.388163 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.389126 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.389521 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.389610 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:42.391914 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:42.391923 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:42.391935 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:42.391961 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:42.393391 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.393775 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.393780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.393781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.393858 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.394042 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.394044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.394045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.394273 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:42.394435 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:42.394562 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 04:34:42.394695 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[17654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:42.404645 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:42.397552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.12.15.254:41893)
0806 04:34:42.397688 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:42.397692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:42.397705 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:42.397773 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:42.397775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:42.397812 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:42.397873 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:42.398381 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.398869 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.399571 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:42.399777 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:42.399840 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:42.402009 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:42.402012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:42.402014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:42.402038 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:42.403116 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.403482 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.403485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.403486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.403558 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:42.403810 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:42.403813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:42.403815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:42.404019 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:42.404137 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:42.404387 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 04:34:42.404496 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 04:34:42.405030 17723 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41174
I20260806 04:34:42.405591 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 17822
I20260806 04:34:42.421805 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 18032
I20260806 04:34:42.432096 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 18237
I20260806 04:34:42.443198 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 17678
2026-08-06T04:34:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2303 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[18456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[18456](info): set up 2 sockets
Aug 06 04:34:42 dist-test-slave-xt4k krb5kdc[18456](info): commencing operation
krb5kdc: starting...
W20260806 04:34:43.586848 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:43 dist-test-slave-xt4k krb5kdc[18456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:43.710928 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:41803
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:41803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:43.823647 18471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:43.823918 18471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:43.823979 18471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:43.828016 18471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:43.828081 18471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:43.828109 18471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:43.828130 18471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:43.828158 18471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:43.833360 18471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:41803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:41803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:43.834379 18471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:43.835372 18471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:43.843694 18477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:43.843720 18479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:43.843694 18476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:43.843747 18471 server_base.cc:1034] running on GCE node
I20260806 04:34:43.844420 18471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:43.845520 18471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:43.846691 18471 hybrid_clock.cc:648] HybridClock initialized: now 1785990883846671 us; error 43 us; skew 500 ppm
Aug 06 04:34:43 dist-test-slave-xt4k krb5kdc[18456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990883, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:43.849366 18471 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:43.850046 18471 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:43.850116 18471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:43.850276 18471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:43.852268 18471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "18bd0bb08a4a4a9ead9b5fde098747a5"
format_stamp: "Formatted at 2026-08-06 04:34:43 on dist-test-slave-xt4k"
server_key: "481cde465cb2cb004bd5b542aeaa74a7"
server_key_iv: "33aa0fee8c3248950adba1bacd8570f2"
server_key_version: "encryptionkey@0"
I20260806 04:34:43.852841 18471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "18bd0bb08a4a4a9ead9b5fde098747a5"
format_stamp: "Formatted at 2026-08-06 04:34:43 on dist-test-slave-xt4k"
server_key: "481cde465cb2cb004bd5b542aeaa74a7"
server_key_iv: "33aa0fee8c3248950adba1bacd8570f2"
server_key_version: "encryptionkey@0"
I20260806 04:34:43.856467 18471 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:43.858925 18487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:43.860154 18471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:43.860260 18471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "18bd0bb08a4a4a9ead9b5fde098747a5"
format_stamp: "Formatted at 2026-08-06 04:34:43 on dist-test-slave-xt4k"
server_key: "481cde465cb2cb004bd5b542aeaa74a7"
server_key_iv: "33aa0fee8c3248950adba1bacd8570f2"
server_key_version: "encryptionkey@0"
I20260806 04:34:43.860376 18471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:43.869481 18471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:43.872519 18471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:43.872690 18471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:43.879621 18471 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:41803
I20260806 04:34:43.879870 18570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:41803 every 8 connection(s)
I20260806 04:34:43.880275 18471 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:43.883086 18571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:43.887686 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 18471
I20260806 04:34:43.887835 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:43.888078 12351 external_mini_cluster.cc:1428] Setting key 6236f46c7698e12a61ff9f6884805e8d
I20260806 04:34:43.888984 18571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5: Bootstrap starting.
I20260806 04:34:43.891273 18571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:43.892225 18571 log.cc:824] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:43.894440 18571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5: No bootstrap required, opened a new log
Aug 06 04:34:43 dist-test-slave-xt4k krb5kdc[18456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:43.898501 18571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41803 } }
I20260806 04:34:43.898733 18571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:43.898808 18571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bd0bb08a4a4a9ead9b5fde098747a5, State: Initialized, Role: FOLLOWER
I20260806 04:34:43.899264 18571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41803 } }
I20260806 04:34:43.899391 18571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:43.899453 18571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:43.899549 18571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:43.900449 18571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41803 } }
I20260806 04:34:43.900784 18571 leader_election.cc:302] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18bd0bb08a4a4a9ead9b5fde098747a5; no voters:
I20260806 04:34:43.901086 18571 leader_election.cc:288] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:43.901180 18577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:43.901391 18577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [term 1 LEADER]: Becoming Leader. State: Replica: 18bd0bb08a4a4a9ead9b5fde098747a5, State: Running, Role: LEADER
I20260806 04:34:43.901782 18577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41803 } }
I20260806 04:34:43.902489 18571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:43.902992 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:43.889548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.12.15.254:41803)
0806 04:34:43.889965 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:43.889981 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:43.890016 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:34:43.890665 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:43.890683 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:43.891012 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:43.891298 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:43.892207 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:43.893393 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:43.894555 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:43.895042 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:43.895170 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:43.897880 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:43.897885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:43.897899 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:43.897928 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:43.899605 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:43.900021 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:43.900027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:43.900033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:43.900152 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:43.900441 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:43.900445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:43.900446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:43.900891 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:43.901120 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:43.901823 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 04:34:43.902117 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260806 04:34:43.903614 18578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18bd0bb08a4a4a9ead9b5fde098747a5. Latest consensus state: current_term: 1 leader_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41803 } } }
I20260806 04:34:43.903731 18578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:43.903026 18577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd0bb08a4a4a9ead9b5fde098747a5" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41803 } } }
I20260806 04:34:43.904078 18577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:43.904183 18590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:43.907626 18590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:43.912859 18590 catalog_manager.cc:1347] Generated new cluster ID: 6b333de427a34f6d9eb5ce01aafb3761
I20260806 04:34:43.912931 18590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:43.928650 18590 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:43.929649 18590 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:43.935284 18590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5: Generated new TSK 0
I20260806 04:34:43.935931 18590 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:43.971140 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41803
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:44.102860 18604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:44.103252 18604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:44.103314 18604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:44.107651 18604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:44.107726 18604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:44.107791 18604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:44.113348 18604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:44.114508 18604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:44.115511 18604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:44.125027 18613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:44.125037 18610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:44.125094 18604 server_base.cc:1034] running on GCE node
W20260806 04:34:44.125044 18617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:44.125602 18604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:44.126199 18604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:44.127354 18604 hybrid_clock.cc:648] HybridClock initialized: now 1785990884127321 us; error 46 us; skew 500 ppm
Aug 06 04:34:44 dist-test-slave-xt4k krb5kdc[18456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:44.130228 18604 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:44.130972 18604 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:44.131033 18604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:44.131201 18604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:44.133288 18604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "e34d7489d0c94092bf773c13c45ead54"
format_stamp: "Formatted at 2026-08-06 04:34:44 on dist-test-slave-xt4k"
server_key: "b7adc9f34e0d6e44d9888687d4e1cb64"
server_key_iv: "ecd881fa1c9588a83320dfdd572f05bf"
server_key_version: "encryptionkey@0"
I20260806 04:34:44.133823 18604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "e34d7489d0c94092bf773c13c45ead54"
format_stamp: "Formatted at 2026-08-06 04:34:44 on dist-test-slave-xt4k"
server_key: "b7adc9f34e0d6e44d9888687d4e1cb64"
server_key_iv: "ecd881fa1c9588a83320dfdd572f05bf"
server_key_version: "encryptionkey@0"
I20260806 04:34:44.137465 18604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 04:34:44.139834 18626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:44.141104 18604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:44.141218 18604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "e34d7489d0c94092bf773c13c45ead54"
format_stamp: "Formatted at 2026-08-06 04:34:44 on dist-test-slave-xt4k"
server_key: "b7adc9f34e0d6e44d9888687d4e1cb64"
server_key_iv: "ecd881fa1c9588a83320dfdd572f05bf"
server_key_version: "encryptionkey@0"
I20260806 04:34:44.141311 18604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:44.161550 18604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:44.164595 18604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:44.164812 18604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:44.165410 18604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:44.166270 18604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:44.166319 18604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:44.166374 18604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:44.166404 18604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:44.175249 18604 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:35541
I20260806 04:34:44.175271 18793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:35541 every 8 connection(s)
I20260806 04:34:44.175829 18604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:44.179257 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 18604
I20260806 04:34:44.179380 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:44.179641 12351 external_mini_cluster.cc:1428] Setting key 9d87e3d96427446ef3a2acadfecbe14e
Aug 06 04:34:44 dist-test-slave-xt4k krb5kdc[18456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:44.190671 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:44.177562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:54521 (local address 127.12.15.254:41803)
0806 04:34:44.177692 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:44.177697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:44.178820 (+ 1123us) server_negotiation.cc:407] Connection header received
0806 04:34:44.179716 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:44.179720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:44.179784 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:44.179879 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:44.181416 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.182120 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.183470 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.183704 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.183781 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:44.186699 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:44.186703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:44.186705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:44.186731 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:44.188139 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:44.188805 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:44.188808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:44.188810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:44.188900 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:44.189332 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:44.189335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:44.189337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:44.189539 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:44.189626 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:44.190359 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 04:34:44.190484 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:34:44.191511 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:44.177893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41803 (local address 127.12.15.193:54521)
0806 04:34:44.178680 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:44.178720 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:44.179507 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:44.180105 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:44.180114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:44.180598 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:44.181267 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:44.181284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.182272 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.182275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:44.183324 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:44.183337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.183849 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.183853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:44.184060 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:44.184671 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:44.184691 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:44.186505 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:44.188268 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:44.188274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:44.188294 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:44.188681 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:44.188997 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:44.189001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:44.189002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:44.189195 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:44.189656 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:44.189664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:44.190042 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 04:34:44.190265 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 04:34:44.190630 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260806 04:34:44.192845 18794 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41803
I20260806 04:34:44.193091 18794 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:34:44.193679 18794 heartbeater.cc:511] Master 127.12.15.254:41803 requested a full tablet report, sending...
I20260806 04:34:44.195300 18509 ts_manager.cc:194] Registered new tserver with Master: e34d7489d0c94092bf773c13c45ead54 (127.12.15.193:35541)
I20260806 04:34:44.196914 18509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:54521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:44.210994 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41803
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:44.327687 18804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:44.327947 18804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:44.327991 18804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:44.332047 18804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:44.332109 18804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:44.332172 18804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:44.337450 18804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:44.338405 18804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:44.339362 18804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:44.348580 18813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:44.348592 18810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:44.348594 18811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:44.348649 18804 server_base.cc:1034] running on GCE node
I20260806 04:34:44.349247 18804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:44.349872 18804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:44.351028 18804 hybrid_clock.cc:648] HybridClock initialized: now 1785990884351011 us; error 28 us; skew 500 ppm
Aug 06 04:34:44 dist-test-slave-xt4k krb5kdc[18456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:44.353710 18804 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:44.354410 18804 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:44.354475 18804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:44.354662 18804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:44.356657 18804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "c0688b3870464f2a89152f853cad479d"
format_stamp: "Formatted at 2026-08-06 04:34:44 on dist-test-slave-xt4k"
server_key: "f6f368d3b9b448006b9c1348fb940df3"
server_key_iv: "11a70f3a2d7c42ecfad5e87ced700a9d"
server_key_version: "encryptionkey@0"
I20260806 04:34:44.357201 18804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "c0688b3870464f2a89152f853cad479d"
format_stamp: "Formatted at 2026-08-06 04:34:44 on dist-test-slave-xt4k"
server_key: "f6f368d3b9b448006b9c1348fb940df3"
server_key_iv: "11a70f3a2d7c42ecfad5e87ced700a9d"
server_key_version: "encryptionkey@0"
I20260806 04:34:44.360654 18804 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:44.362893 18822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:44.364106 18804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:44.364208 18804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "c0688b3870464f2a89152f853cad479d"
format_stamp: "Formatted at 2026-08-06 04:34:44 on dist-test-slave-xt4k"
server_key: "f6f368d3b9b448006b9c1348fb940df3"
server_key_iv: "11a70f3a2d7c42ecfad5e87ced700a9d"
server_key_version: "encryptionkey@0"
I20260806 04:34:44.364310 18804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:44.381816 18804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:44.384627 18804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:44.384776 18804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:44.385300 18804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:44.386082 18804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:44.386137 18804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:44.386178 18804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:44.386197 18804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:44.394517 18804 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:40609
I20260806 04:34:44.394537 18993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:40609 every 8 connection(s)
I20260806 04:34:44.395109 18804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:44.398094 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 18804
I20260806 04:34:44.398204 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:44.398491 12351 external_mini_cluster.cc:1428] Setting key dcd942f9939e622a41b63962d1be27d9
Aug 06 04:34:44 dist-test-slave-xt4k krb5kdc[18456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990884, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:44.408954 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:44.396764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:44621 (local address 127.12.15.254:41803)
0806 04:34:44.396920 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:44.396925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:44.398060 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 04:34:44.399030 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:44.399033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:44.399087 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:44.399168 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:44.400492 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.401015 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.402247 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.402569 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.402668 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:44.405497 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:44.405501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:44.405503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:44.405531 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:44.406923 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:44.407393 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:44.407397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:44.407399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:44.407479 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:44.407795 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:44.407798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:44.407801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:44.408010 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:44.408098 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:44.408654 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 04:34:44.408786 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:34:44.409817 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:44.397154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41803 (local address 127.12.15.194:44621)
0806 04:34:44.397915 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:44.397953 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:44.398786 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:44.399346 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:44.399354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:44.399820 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:44.400327 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:44.400374 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.401141 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.401144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:44.402132 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:44.402143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:44.402710 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:44.402714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:44.402901 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:44.403465 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:44.403481 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:44.405329 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:44.407029 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:44.407035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:44.407049 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:44.407299 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:44.407560 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:44.407563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:44.407565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:44.407706 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:44.408116 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:44.408123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:44.408407 (+ 284us) client_negotiation.cc:769] Sending connection context
0806 04:34:44.408585 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 04:34:44.408900 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:44.415081 18997 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41803
I20260806 04:34:44.415325 18997 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:44.415834 18997 heartbeater.cc:511] Master 127.12.15.254:41803 requested a full tablet report, sending...
I20260806 04:34:44.416918 18509 ts_manager.cc:194] Registered new tserver with Master: c0688b3870464f2a89152f853cad479d (127.12.15.194:40609)
I20260806 04:34:44.417502 18509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:44621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:34:45.199471 18794 heartbeater.cc:503] Master 127.12.15.254:41803 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
W20260806 04:34:45.326040 12351 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.12.15.195: real 0.926s user 0.000s sys 0.004s
I20260806 04:34:45.326614 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41803
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:34:45.419996 18997 heartbeater.cc:503] Master 127.12.15.254:41803 was elected leader, sending a full tablet report...
W20260806 04:34:45.444846 19004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:45.445133 19004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:45.445181 19004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:45.449319 19004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:45.449384 19004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:45.449452 19004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:45.455090 19004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:45.456203 19004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:45.457336 19004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:45.466946 19010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:45.467008 19014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:45.467005 19011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:45.467247 19004 server_base.cc:1034] running on GCE node
I20260806 04:34:45.467701 19004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:45.468377 19004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:45.469558 19004 hybrid_clock.cc:648] HybridClock initialized: now 1785990885469532 us; error 43 us; skew 500 ppm
Aug 06 04:34:45 dist-test-slave-xt4k krb5kdc[18456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:45.472601 19004 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:45.473395 19004 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:45.473464 19004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:45.473661 19004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:45.475697 19004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "e6182beab92844bd993f093ad946e232"
format_stamp: "Formatted at 2026-08-06 04:34:45 on dist-test-slave-xt4k"
server_key: "de7f5f63071c497c96daafe36a1a3d14"
server_key_iv: "62e224b9dc7c0207075985adc3d42d30"
server_key_version: "encryptionkey@0"
I20260806 04:34:45.476250 19004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "e6182beab92844bd993f093ad946e232"
format_stamp: "Formatted at 2026-08-06 04:34:45 on dist-test-slave-xt4k"
server_key: "de7f5f63071c497c96daafe36a1a3d14"
server_key_iv: "62e224b9dc7c0207075985adc3d42d30"
server_key_version: "encryptionkey@0"
I20260806 04:34:45.480041 19004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:34:45.482597 19028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:45.483865 19004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:45.483983 19004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "e6182beab92844bd993f093ad946e232"
format_stamp: "Formatted at 2026-08-06 04:34:45 on dist-test-slave-xt4k"
server_key: "de7f5f63071c497c96daafe36a1a3d14"
server_key_iv: "62e224b9dc7c0207075985adc3d42d30"
server_key_version: "encryptionkey@0"
I20260806 04:34:45.484107 19004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:45.496273 19004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:45.499445 19004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:45.499639 19004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:45.500255 19004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:45.501143 19004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:45.501192 19004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:45.501246 19004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:45.501273 19004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:45.510313 19004 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39203
I20260806 04:34:45.510330 19199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39203 every 8 connection(s)
I20260806 04:34:45.510936 19004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:45.513734 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 19004
I20260806 04:34:45.513880 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:45.514324 12351 external_mini_cluster.cc:1428] Setting key f45575492d366356bcf085c94030173e
Aug 06 04:34:45 dist-test-slave-xt4k krb5kdc[18456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990885, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:45.525532 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.512796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:52787 (local address 127.12.15.254:41803)
0806 04:34:45.513055 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.513059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.514294 (+ 1235us) server_negotiation.cc:407] Connection header received
0806 04:34:45.515356 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.515360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.515421 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.515520 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:45.517030 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.517628 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.518754 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.518992 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.519067 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.521576 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:45.521580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:45.521582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:45.521616 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:45.523111 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:45.523759 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:45.523762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:45.523763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:45.523858 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:45.524285 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:45.524287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:45.524289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:45.524517 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:45.524600 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.525266 (+ 666us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.525388 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260806 04:34:45.526840 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.513205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41803 (local address 127.12.15.195:52787)
0806 04:34:45.514116 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.514161 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:45.515119 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:45.515774 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:45.515782 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:45.516294 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:45.516895 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.516909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.517763 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.517766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.518660 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.518670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.519126 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.519129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.519276 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.519840 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:45.519857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:45.521385 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:45.523237 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:45.523244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:45.523261 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:45.523607 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:45.523982 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:45.523985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:45.523988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:45.524189 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:45.524635 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:45.524641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:45.524993 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 04:34:45.525202 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 04:34:45.525546 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":110,"threads_started":1}
I20260806 04:34:45.528023 19201 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41803
I20260806 04:34:45.528270 19201 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:45.528851 19201 heartbeater.cc:511] Master 127.12.15.254:41803 requested a full tablet report, sending...
I20260806 04:34:45.529886 18508 ts_manager.cc:194] Registered new tserver with Master: e6182beab92844bd993f093ad946e232 (127.12.15.195:39203)
I20260806 04:34:45.530468 18508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:52787
I20260806 04:34:45.540877 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:45.550485 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.543395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.12.15.254:41803)
0806 04:34:45.543570 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.543575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.543688 (+ 113us) server_negotiation.cc:407] Connection header received
0806 04:34:45.543788 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.543790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.543833 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.543899 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:45.544691 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.545279 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.546139 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.546371 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.546468 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.547844 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:45.547847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:45.547850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:45.547875 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:45.549064 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:45.549432 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:45.549435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:45.549436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:45.549521 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:45.549704 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:45.549705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:45.549707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:45.549908 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:45.550014 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.550205 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.550309 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:34:45.553547 18508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:45.556931 18508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:45.572355 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.566350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.12.15.194:40609)
0806 04:34:45.566737 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.566744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.566767 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:34:45.566997 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.567003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.567229 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.567885 (+ 656us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:45.568047 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.569194 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.571019 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.571652 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.571752 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.571912 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.572079 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.572174 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":95,"threads_started":1}
I20260806 04:34:45.572329 19220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.566202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:40609 (local address 127.0.0.1:53370)
0806 04:34:45.566613 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.566668 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:45.566798 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:45.567408 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:45.567415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:45.567442 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:45.567593 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.567601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.569345 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.569349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.570885 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.570896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.571766 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.571769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.571883 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.571940 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 04:34:45.572039 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 04:34:45.572175 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":192,"thread_start_us":125,"threads_started":1}
I20260806 04:34:45.572748 19222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.566233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39203 (local address 127.0.0.1:38910)
0806 04:34:45.567024 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.567038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:45.567158 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:45.568007 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:45.568011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:45.568031 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:45.568151 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.568157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.569255 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.569258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.570913 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.570921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.572016 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.572028 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.572126 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.572153 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 04:34:45.572201 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 04:34:45.572272 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":121,"thread_start_us":117,"threads_started":1}
I20260806 04:34:45.572875 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.567097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.12.15.195:39203)
0806 04:34:45.567551 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.567563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.567586 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:34:45.567660 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.567665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.567851 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.567991 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:45.568255 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.569147 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.571060 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.571829 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.572424 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.572574 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.572623 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.572715 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":155,"threads_started":1}
I20260806 04:34:45.572978 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.567002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35541 (local address 127.0.0.1:35918)
0806 04:34:45.567789 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.567802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:45.567886 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:45.568831 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:45.568834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:45.568854 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:45.568949 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.568954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.570003 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.570007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.571095 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.571125 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.572330 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.572367 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.572601 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.572623 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:34:45.572690 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 04:34:45.572750 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":49,"thread_start_us":92,"threads_started":1}
I20260806 04:34:45.575752 18899 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3c871083264c49ad7f3bf884d76eef (DEFAULT_TABLE table=test-table [id=b63b768126d849fbab4d220fcb580bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:45.575764 19098 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3c871083264c49ad7f3bf884d76eef (DEFAULT_TABLE table=test-table [id=b63b768126d849fbab4d220fcb580bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:45.576804 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.567442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.12.15.193:35541)
0806 04:34:45.568417 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.568424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.568445 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:34:45.568512 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.568517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.568698 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.568831 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:45.569062 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.569885 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.571237 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.572047 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.572137 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.572272 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.576556 (+ 4284us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.576677 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":107,"threads_started":1}
I20260806 04:34:45.577001 19098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3c871083264c49ad7f3bf884d76eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:45.577826 18899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3c871083264c49ad7f3bf884d76eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:45.579828 18701 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3c871083264c49ad7f3bf884d76eef (DEFAULT_TABLE table=test-table [id=b63b768126d849fbab4d220fcb580bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:45.580965 18701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3c871083264c49ad7f3bf884d76eef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:45.583786 19230 tablet_bootstrap.cc:492] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Bootstrap starting.
I20260806 04:34:45.584568 19231 tablet_bootstrap.cc:492] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Bootstrap starting.
I20260806 04:34:45.586851 19230 tablet_bootstrap.cc:654] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:45.586851 19231 tablet_bootstrap.cc:654] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:45.587219 19232 tablet_bootstrap.cc:492] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: Bootstrap starting.
I20260806 04:34:45.588204 19231 log.cc:824] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:45.588615 19230 log.cc:824] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:45.589406 19232 tablet_bootstrap.cc:654] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:45.590204 19232 log.cc:824] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:45.590278 19230 tablet_bootstrap.cc:492] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: No bootstrap required, opened a new log
I20260806 04:34:45.590386 19231 tablet_bootstrap.cc:492] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: No bootstrap required, opened a new log
I20260806 04:34:45.590451 19230 ts_tablet_manager.cc:1397] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 04:34:45.590584 19231 ts_tablet_manager.cc:1397] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:34:45.593346 19231 raft_consensus.cc:348] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.593559 19231 raft_consensus.cc:374] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:45.593593 19231 raft_consensus.cc:729] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6182beab92844bd993f093ad946e232, State: Initialized, Role: FOLLOWER
I20260806 04:34:45.593698 19230 raft_consensus.cc:348] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.593950 19230 raft_consensus.cc:374] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:45.594007 19230 raft_consensus.cc:729] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0688b3870464f2a89152f853cad479d, State: Initialized, Role: FOLLOWER
I20260806 04:34:45.594043 19231 consensus_queue.cc:260] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.594556 19230 consensus_queue.cc:260] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.595047 19231 ts_tablet_manager.cc:1428] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:45.595273 19232 tablet_bootstrap.cc:492] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: No bootstrap required, opened a new log
I20260806 04:34:45.595397 19201 heartbeater.cc:503] Master 127.12.15.254:41803 was elected leader, sending a full tablet report...
I20260806 04:34:45.595469 19232 ts_tablet_manager.cc:1397] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 04:34:45.595721 19230 ts_tablet_manager.cc:1428] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 04:34:45.599583 19232 raft_consensus.cc:348] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.599841 19232 raft_consensus.cc:374] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:45.599900 19232 raft_consensus.cc:729] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34d7489d0c94092bf773c13c45ead54, State: Initialized, Role: FOLLOWER
I20260806 04:34:45.600252 19232 consensus_queue.cc:260] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.601099 19232 ts_tablet_manager.cc:1428] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 04:34:45.606065 19240 raft_consensus.cc:482] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:45.606202 19240 raft_consensus.cc:504] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.607138 19240 leader_election.cc:288] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0688b3870464f2a89152f853cad479d (127.12.15.194:40609), e34d7489d0c94092bf773c13c45ead54 (127.12.15.193:35541)
I20260806 04:34:45.610905 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.607719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:56969 (local address 127.12.15.193:35541)
0806 04:34:45.607866 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.607870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.608044 (+ 174us) server_negotiation.cc:407] Connection header received
0806 04:34:45.608138 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.608141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.608181 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.608241 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:45.608516 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.608993 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.610036 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.610421 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.610520 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.610573 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.610727 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.610795 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 04:34:45.610927 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.607607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35541 (local address 127.12.15.195:56969)
0806 04:34:45.607963 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.607975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:45.608064 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:45.608257 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:45.608260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:45.608275 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:45.608413 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.608420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.609090 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.609093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.609921 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.609931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.610521 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.610524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.610618 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.610642 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:34:45.610705 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 04:34:45.610775 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":30,"thread_start_us":109,"threads_started":1}
I20260806 04:34:45.611184 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.607327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:40609 (local address 127.12.15.195:33301)
0806 04:34:45.607441 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.607456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:45.607540 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:45.607795 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:45.607798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:45.607818 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:45.607945 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.607952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.608724 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.608727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.609609 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:45.609619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.610806 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.610809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:45.610912 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.610932 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:34:45.610993 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 04:34:45.611068 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":19}
I20260806 04:34:45.611457 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3c871083264c49ad7f3bf884d76eef" candidate_uuid: "e6182beab92844bd993f093ad946e232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e34d7489d0c94092bf773c13c45ead54" is_pre_election: true
I20260806 04:34:45.611507 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.607414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33301 (local address 127.12.15.194:40609)
0806 04:34:45.607521 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.607525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.607537 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:34:45.607634 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.607637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.607686 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.607865 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:45.608051 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.608621 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.609724 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.610697 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.610806 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.610868 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.611336 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.611421 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 04:34:45.611729 18727 raft_consensus.cc:2454] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6182beab92844bd993f093ad946e232 in term 0.
I20260806 04:34:45.611995 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3c871083264c49ad7f3bf884d76eef" candidate_uuid: "e6182beab92844bd993f093ad946e232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0688b3870464f2a89152f853cad479d" is_pre_election: true
I20260806 04:34:45.612203 19031 leader_election.cc:302] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e34d7489d0c94092bf773c13c45ead54, e6182beab92844bd993f093ad946e232; no voters:
I20260806 04:34:45.612234 18923 raft_consensus.cc:2454] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6182beab92844bd993f093ad946e232 in term 0.
I20260806 04:34:45.612493 19240 raft_consensus.cc:2781] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:45.612581 19240 raft_consensus.cc:482] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:45.612632 19240 raft_consensus.cc:3037] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:45.624670 19240 raft_consensus.cc:504] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.625164 19240 leader_election.cc:288] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [CANDIDATE]: Term 1 election: Requested vote from peers c0688b3870464f2a89152f853cad479d (127.12.15.194:40609), e34d7489d0c94092bf773c13c45ead54 (127.12.15.193:35541)
I20260806 04:34:45.625731 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3c871083264c49ad7f3bf884d76eef" candidate_uuid: "e6182beab92844bd993f093ad946e232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e34d7489d0c94092bf773c13c45ead54"
I20260806 04:34:45.625744 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3c871083264c49ad7f3bf884d76eef" candidate_uuid: "e6182beab92844bd993f093ad946e232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0688b3870464f2a89152f853cad479d"
I20260806 04:34:45.625874 18923 raft_consensus.cc:3037] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:45.625880 18727 raft_consensus.cc:3037] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:45.627167 18727 raft_consensus.cc:2454] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6182beab92844bd993f093ad946e232 in term 1.
I20260806 04:34:45.627215 18923 raft_consensus.cc:2454] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6182beab92844bd993f093ad946e232 in term 1.
I20260806 04:34:45.627516 19031 leader_election.cc:302] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e34d7489d0c94092bf773c13c45ead54, e6182beab92844bd993f093ad946e232; no voters:
I20260806 04:34:45.627713 19240 raft_consensus.cc:2781] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:45.627948 19240 raft_consensus.cc:686] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 1 LEADER]: Becoming Leader. State: Replica: e6182beab92844bd993f093ad946e232, State: Running, Role: LEADER
I20260806 04:34:45.628399 19240 consensus_queue.cc:237] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } }
I20260806 04:34:45.631603 18508 catalog_manager.cc:5495] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6182beab92844bd993f093ad946e232 (127.12.15.195). New cstate: current_term: 1 leader_uuid: "e6182beab92844bd993f093ad946e232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6182beab92844bd993f093ad946e232" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 } health_report { overall_health: UNKNOWN } } }
W20260806 04:34:45.646699 18998 tablet.cc:2369] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:45.650467 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.646983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.12.15.195:39203)
0806 04:34:45.647101 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.647106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.647121 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:34:45.647273 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.647277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.647334 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.647422 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:45.647682 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.648192 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.649060 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:45.649287 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:45.649374 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:45.649668 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:45.650079 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:45.650185 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.650255 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.650312 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 04:34:45.656469 18727 raft_consensus.cc:1261] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54 [term 1 FOLLOWER]: Refusing update from remote peer e6182beab92844bd993f093ad946e232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:45.656469 18923 raft_consensus.cc:1261] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 1 FOLLOWER]: Refusing update from remote peer e6182beab92844bd993f093ad946e232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:45.657192 19240 consensus_queue.cc:1035] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e34d7489d0c94092bf773c13c45ead54" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 35541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:45.657382 19245 consensus_queue.cc:1035] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0688b3870464f2a89152f853cad479d" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:45.666107 19249 mvcc.cc:205] Tried to move back new op lower bound from 7315418667641208832 to 7315418667536875520. Current Snapshot: MvccSnapshot[applied={T|T < 7315418667641208832}]
I20260806 04:34:45.673597 18508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:45.673766 18508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:45.676153 18508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18bd0bb08a4a4a9ead9b5fde098747a5: Sending DeleteTablet for 3 replicas of tablet 1c3c871083264c49ad7f3bf884d76eef
I20260806 04:34:45.677014 19098 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3c871083264c49ad7f3bf884d76eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:45 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:38910
I20260806 04:34:45.677191 18701 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3c871083264c49ad7f3bf884d76eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:45 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:35918
I20260806 04:34:45.677543 18899 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3c871083264c49ad7f3bf884d76eef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:45 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:53370
I20260806 04:34:45.677707 19270 tablet_replica.cc:330] T 1c3c871083264c49ad7f3bf884d76eef P e34d7489d0c94092bf773c13c45ead54: stopping tablet replica
I20260806 04:34:45.678033 19271 tablet_replica.cc:330] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: stopping tablet replica
I20260806 04:34:45.678352 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 18604
I20260806 04:34:45.678368 19271 raft_consensus.cc:2229] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:45.683019 19271 raft_consensus.cc:2258] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:45.685979 19271 ts_tablet_manager.cc:1905] T 1c3c871083264c49ad7f3bf884d76eef P c0688b3870464f2a89152f853cad479d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 04:34:45.691843 18490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:35541 (error 108)
I20260806 04:34:45.691919 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 18804
I20260806 04:34:45.692076 19268 tablet_replica.cc:330] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: stopping tablet replica
I20260806 04:34:45.692735 19268 raft_consensus.cc:2229] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:45.693225 19268 raft_consensus.cc:2258] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:45.697316 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.696990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35541 (local address 127.0.0.1:35924)
0806 04:34:45.697139 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.697216 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:35541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 04:34:45.697575 18490 catalog_manager.cc:4584] TS e34d7489d0c94092bf773c13c45ead54 (127.12.15.193:35541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c3c871083264c49ad7f3bf884d76eef: Network error: Client connection negotiation failed: client connection to 127.12.15.193:35541: connect: Connection refused (error 111)
I20260806 04:34:45.697656 19268 ts_tablet_manager.cc:1905] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:45.728144 19268 ts_tablet_manager.cc:1918] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:45.728430 19268 log.cc:1196] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/1c3c871083264c49ad7f3bf884d76eef
I20260806 04:34:45.728883 19268 ts_tablet_manager.cc:1939] T 1c3c871083264c49ad7f3bf884d76eef P e6182beab92844bd993f093ad946e232: Deleting consensus metadata
I20260806 04:34:45.732633 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 19004
I20260806 04:34:45.732872 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.732592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:40609 (local address 127.0.0.1:53386)
0806 04:34:45.732724 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.732778 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.194:40609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 04:34:45.751118 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.750802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35541 (local address 127.0.0.1:35940)
0806 04:34:45.750966 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.751027 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:35541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 04:34:45.751804 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.751598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:35541 (local address 127.0.0.1:35946)
0806 04:34:45.751700 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:45.751747 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:35541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 04:34:45.753805 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 18471
2026-08-06T04:34:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3308 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T04:34:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:45Z Disabled control of system clock
I20260806 04:34:45.789999 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:44657
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:35389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:44657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 04:34:45.905553 19283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:45.905812 19283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:45.905848 19283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:45.909960 19283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:45.910027 19283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:45.910040 19283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:45.910056 19283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:45.910072 19283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:45.915627 19283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:44657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:44657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:45.916797 19283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:45.918238 19283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:45.927107 19290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:45.927345 19289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:45.927114 19294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:45.927506 19283 server_base.cc:1034] running on GCE node
I20260806 04:34:45.928076 19283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:45.929252 19283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:45.932574 19283 hybrid_clock.cc:648] HybridClock initialized: now 1785990885932542 us; error 35 us; skew 500 ppm
I20260806 04:34:45.934614 19283 webserver.cc:460] Webserver started at http://127.12.15.254:41567/ using document root <none> and password file <none>
I20260806 04:34:45.935281 19283 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:45.935359 19283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:45.935628 19283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:45.937814 19283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "724faf9d039d486aa64439600a2ec69c"
format_stamp: "Formatted at 2026-08-06 04:34:45 on dist-test-slave-xt4k"
server_key: "09c45fc2459e4d6b41d46a4bc1e939c7"
server_key_iv: "e533030244c4a3e9fad9779d18f6bcae"
server_key_version: "encryptionkey@0"
I20260806 04:34:45.938416 19283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "724faf9d039d486aa64439600a2ec69c"
format_stamp: "Formatted at 2026-08-06 04:34:45 on dist-test-slave-xt4k"
server_key: "09c45fc2459e4d6b41d46a4bc1e939c7"
server_key_iv: "e533030244c4a3e9fad9779d18f6bcae"
server_key_version: "encryptionkey@0"
I20260806 04:34:45.942222 19283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:45.944700 19302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:45.946121 19283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:45.946233 19283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "724faf9d039d486aa64439600a2ec69c"
format_stamp: "Formatted at 2026-08-06 04:34:45 on dist-test-slave-xt4k"
server_key: "09c45fc2459e4d6b41d46a4bc1e939c7"
server_key_iv: "e533030244c4a3e9fad9779d18f6bcae"
server_key_version: "encryptionkey@0"
I20260806 04:34:45.946326 19283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:45.951084 19283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:45.951686 19283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:45.951834 19283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:45.958714 19283 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:44657
I20260806 04:34:45.958732 19373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:44657 every 8 connection(s)
I20260806 04:34:45.959674 19283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:45.962996 19376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:45.966138 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 19283
I20260806 04:34:45.966248 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:45.966537 12351 external_mini_cluster.cc:1428] Setting key 23ee75e86fb467416bfe4061ebc313ed
I20260806 04:34:45.969089 19376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c: Bootstrap starting.
I20260806 04:34:45.971531 19376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:45.972126 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:45.967876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.12.15.254:44657)
0806 04:34:45.968576 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:45.968597 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:45.968644 (+ 47us) server_negotiation.cc:407] Connection header received
0806 04:34:45.969331 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:45.969352 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:45.969719 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:45.970039 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:45.970797 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:45.970802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:45.970814 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:45.970825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:45.970881 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:45.970933 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:45.971043 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:45.971170 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 04:34:45.971411 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
I20260806 04:34:45.972285 19376 log.cc:824] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:45.974295 19376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c: No bootstrap required, opened a new log
I20260806 04:34:45.977861 19376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724faf9d039d486aa64439600a2ec69c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 44657 } }
I20260806 04:34:45.978067 19376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:45.978101 19376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724faf9d039d486aa64439600a2ec69c, State: Initialized, Role: FOLLOWER
I20260806 04:34:45.978487 19376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724faf9d039d486aa64439600a2ec69c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 44657 } }
I20260806 04:34:45.978607 19376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:45.978652 19376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:45.978721 19376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:46.028033 19376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724faf9d039d486aa64439600a2ec69c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 44657 } }
I20260806 04:34:46.028625 19376 leader_election.cc:302] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 724faf9d039d486aa64439600a2ec69c; no voters:
I20260806 04:34:46.029033 19376 leader_election.cc:288] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:46.029179 19388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:46.029428 19388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [term 1 LEADER]: Becoming Leader. State: Replica: 724faf9d039d486aa64439600a2ec69c, State: Running, Role: LEADER
I20260806 04:34:46.029812 19388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724faf9d039d486aa64439600a2ec69c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 44657 } }
I20260806 04:34:46.030474 19376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:46.031697 19392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 724faf9d039d486aa64439600a2ec69c. Latest consensus state: current_term: 1 leader_uuid: "724faf9d039d486aa64439600a2ec69c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724faf9d039d486aa64439600a2ec69c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 44657 } } }
I20260806 04:34:46.031742 19391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "724faf9d039d486aa64439600a2ec69c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724faf9d039d486aa64439600a2ec69c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 44657 } } }
I20260806 04:34:46.031845 19392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:46.031852 19391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:46.032593 19402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:46.035663 19402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:46.041867 19402 catalog_manager.cc:1347] Generated new cluster ID: 5ff59d2b0937471d835ec0c740c5a24d
I20260806 04:34:46.042183 19402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:46.063356 19402 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:46.063690 19402 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:46.069224 19402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c: Generated new TSK 0
I20260806 04:34:46.069903 19402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:34:46.090103 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:44657
--builtin_ntp_servers=127.12.15.212:35389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 04:34:46.210423 19419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:46.210686 19419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:46.210727 19419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:46.214967 19419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:46.215029 19419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:46.215096 19419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:46.219954 19419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:44657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:46.221100 19419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:46.222522 19419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:46.232023 19425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:46.232036 19427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:46.232036 19429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:46.232311 19419 server_base.cc:1034] running on GCE node
I20260806 04:34:46.232714 19419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:46.233315 19419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:46.234472 19419 hybrid_clock.cc:648] HybridClock initialized: now 1785990886234459 us; error 32 us; skew 500 ppm
I20260806 04:34:46.236260 19419 webserver.cc:460] Webserver started at http://127.12.15.193:39279/ using document root <none> and password file <none>
I20260806 04:34:46.236860 19419 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:46.236922 19419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:46.237123 19419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:46.239182 19419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "26cdf6a36bf54bd31e3679b5788c4e35"
server_key_iv: "548f1a5fc3be91b397c50481efb52af0"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.239738 19419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "26cdf6a36bf54bd31e3679b5788c4e35"
server_key_iv: "548f1a5fc3be91b397c50481efb52af0"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.243568 19419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:34:46.246157 19438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.247408 19419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:46.247521 19419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "26cdf6a36bf54bd31e3679b5788c4e35"
server_key_iv: "548f1a5fc3be91b397c50481efb52af0"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.247637 19419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:46.252084 19419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:46.252751 19419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:46.252912 19419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:46.253435 19419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:46.254340 19419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:46.254388 19419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.254438 19419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:46.254464 19419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.262521 19419 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38849
I20260806 04:34:46.262571 19606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38849 every 8 connection(s)
I20260806 04:34:46.263412 19419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:46.267158 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 19419
I20260806 04:34:46.267292 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:46.267581 12351 external_mini_cluster.cc:1428] Setting key 0ce7dc8941df61f9341c539f52a6641f
I20260806 04:34:46.269593 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:44657
--builtin_ntp_servers=127.12.15.212:35389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 04:34:46.270110 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.265371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:36399 (local address 127.12.15.254:44657)
0806 04:34:46.265513 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.265518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.266640 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 04:34:46.268361 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.268365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.268410 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.268488 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.269689 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.269693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.269695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.269708 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.269732 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.269741 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.269822 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.269945 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.269984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 04:34:46.271001 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.265813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:44657 (local address 127.12.15.193:36399)
0806 04:34:46.266499 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.266539 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.267930 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.268722 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.268730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.268842 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.269408 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.269410 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.269455 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.269532 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.269817 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.269826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.269832 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.269978 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.270174 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260806 04:34:46.272121 19607 heartbeater.cc:347] Connected to a master server at 127.12.15.254:44657
I20260806 04:34:46.272395 19607 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:46.272796 19607 heartbeater.cc:511] Master 127.12.15.254:44657 requested a full tablet report, sending...
I20260806 04:34:46.274390 19325 ts_manager.cc:194] Registered new tserver with Master: c0d60b0c6e0d45d288c2c5e0f43bf5e6 (127.12.15.193:38849)
W20260806 04:34:46.382200 19611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:46.382453 19611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:46.382493 19611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:46.386399 19611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:46.386447 19611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:46.386499 19611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:46.391113 19611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:44657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:46.392073 19611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:46.393375 19611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:46.402585 19622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:46.402594 19611 server_base.cc:1034] running on GCE node
W20260806 04:34:46.402592 19626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:46.402601 19621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:46.403198 19611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:46.403795 19611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:46.404958 19611 hybrid_clock.cc:648] HybridClock initialized: now 1785990886404927 us; error 40 us; skew 500 ppm
I20260806 04:34:46.406713 19611 webserver.cc:460] Webserver started at http://127.12.15.194:43929/ using document root <none> and password file <none>
I20260806 04:34:46.407291 19611 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:46.407359 19611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:46.407552 19611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:46.409691 19611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "b075cdcc0bf1480fa38606827a763a77"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "17ec80ddd3aed6edfe773b9723ed17f5"
server_key_iv: "1689c34cf51bceb27bb9dbe38f480391"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.410235 19611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "b075cdcc0bf1480fa38606827a763a77"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "17ec80ddd3aed6edfe773b9723ed17f5"
server_key_iv: "1689c34cf51bceb27bb9dbe38f480391"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.413875 19611 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:34:46.416324 19632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.417500 19611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:46.417608 19611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "b075cdcc0bf1480fa38606827a763a77"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "17ec80ddd3aed6edfe773b9723ed17f5"
server_key_iv: "1689c34cf51bceb27bb9dbe38f480391"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.417706 19611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:46.422041 19611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:46.422638 19611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:46.422788 19611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:46.423298 19611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:46.424175 19611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:46.424221 19611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.424270 19611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:46.424296 19611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.432888 19611 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:36639
I20260806 04:34:46.432914 19794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:36639 every 8 connection(s)
I20260806 04:34:46.433761 19611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:46.437466 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 19611
I20260806 04:34:46.437570 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:46.437825 12351 external_mini_cluster.cc:1428] Setting key 3dc6aaf7f984fcc7d45d11bd09c73ddf
I20260806 04:34:46.439761 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.435583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:51449 (local address 127.12.15.254:44657)
0806 04:34:46.435764 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.435768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.436866 (+ 1098us) server_negotiation.cc:407] Connection header received
0806 04:34:46.438190 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.438193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.438233 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.438305 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.439347 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.439351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.439354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.439367 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.439389 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.439400 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.439488 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.439583 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.439627 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:34:46.439823 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:44657
--builtin_ntp_servers=127.12.15.212:35389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 04:34:46.440645 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.436021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:44657 (local address 127.12.15.194:51449)
0806 04:34:46.436726 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.436767 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.437984 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.438477 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.438484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.438581 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.439099 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.439100 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.439140 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.439208 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.439473 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.439478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.439482 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.439614 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.439807 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260806 04:34:46.441828 19795 heartbeater.cc:347] Connected to a master server at 127.12.15.254:44657
I20260806 04:34:46.442056 19795 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:46.442462 19795 heartbeater.cc:511] Master 127.12.15.254:44657 requested a full tablet report, sending...
I20260806 04:34:46.443475 19325 ts_manager.cc:194] Registered new tserver with Master: b075cdcc0bf1480fa38606827a763a77 (127.12.15.194:36639)
W20260806 04:34:46.552919 19799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:46.553197 19799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:46.553253 19799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:46.557264 19799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:46.557314 19799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:46.557391 19799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:46.562132 19799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:44657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:46.563225 19799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:46.564759 19799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:46.574330 19811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:46.574290 19808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:46.574311 19807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:46.574388 19799 server_base.cc:1034] running on GCE node
I20260806 04:34:46.574946 19799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:46.575582 19799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:46.576740 19799 hybrid_clock.cc:648] HybridClock initialized: now 1785990886576721 us; error 35 us; skew 500 ppm
I20260806 04:34:46.581038 19799 webserver.cc:460] Webserver started at http://127.12.15.195:39915/ using document root <none> and password file <none>
I20260806 04:34:46.581624 19799 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:46.581696 19799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:46.581908 19799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:46.583962 19799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "2143cd71429d4188acf66b1ca67e7512"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "98e37f5c0e385de34d92586bcae1abd2"
server_key_iv: "ce08b92d3a6a34071030320e2f6db22a"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.584559 19799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "2143cd71429d4188acf66b1ca67e7512"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "98e37f5c0e385de34d92586bcae1abd2"
server_key_iv: "ce08b92d3a6a34071030320e2f6db22a"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.588296 19799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:46.590814 19819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.592100 19799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:46.592226 19799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "2143cd71429d4188acf66b1ca67e7512"
format_stamp: "Formatted at 2026-08-06 04:34:46 on dist-test-slave-xt4k"
server_key: "98e37f5c0e385de34d92586bcae1abd2"
server_key_iv: "ce08b92d3a6a34071030320e2f6db22a"
server_key_version: "encryptionkey@0"
I20260806 04:34:46.592327 19799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:46.596792 19799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:46.597388 19799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:46.597538 19799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:46.598054 19799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:46.599020 19799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:46.599092 19799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.599143 19799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:46.599170 19799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:46.607342 19799 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:33379
I20260806 04:34:46.607365 19990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:33379 every 8 connection(s)
I20260806 04:34:46.608215 19799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:46.614059 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.610035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:51747 (local address 127.12.15.254:44657)
0806 04:34:46.610183 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.610187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.611276 (+ 1089us) server_negotiation.cc:407] Connection header received
0806 04:34:46.612596 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.612598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.612636 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.612707 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.613713 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.613716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.613718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.613730 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.613749 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.613757 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.613829 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.613914 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.613950 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:34:46.614918 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.610438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:44657 (local address 127.12.15.195:51747)
0806 04:34:46.611137 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.611178 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.612382 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.612840 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.612846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.612933 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.613438 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.613440 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.613507 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.613574 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.613819 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.613823 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.613827 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.613946 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.614142 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260806 04:34:46.615984 19992 heartbeater.cc:347] Connected to a master server at 127.12.15.254:44657
I20260806 04:34:46.616194 19992 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:46.616624 19992 heartbeater.cc:511] Master 127.12.15.254:44657 requested a full tablet report, sending...
I20260806 04:34:46.617247 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 19799
I20260806 04:34:46.617357 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:46.617610 12351 external_mini_cluster.cc:1428] Setting key b2c95576241277c967b87241e0cb81f8
I20260806 04:34:46.617617 19325 ts_manager.cc:194] Registered new tserver with Master: 2143cd71429d4188acf66b1ca67e7512 (127.12.15.195:33379)
I20260806 04:34:46.619553 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:46.622953 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.621450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.12.15.254:44657)
0806 04:34:46.621606 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.621610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.621728 (+ 118us) server_negotiation.cc:407] Connection header received
0806 04:34:46.621824 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.621826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.621857 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.621918 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.622484 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.622488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.622491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.622510 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.622531 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.622539 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.622608 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.622787 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.622827 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 04:34:46.625716 19325 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:46.629150 19325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:46.640234 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.637432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38849 (local address 127.0.0.1:35118)
0806 04:34:46.638067 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.638115 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.638230 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.638737 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.638743 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.638820 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.639411 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.639413 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.639434 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.639502 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.639904 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.639914 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.639920 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.640034 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.640100 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":53,"threads_started":1}
I20260806 04:34:46.640450 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.637593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35118 (local address 127.12.15.193:38849)
0806 04:34:46.637910 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.637923 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.638233 (+ 310us) server_negotiation.cc:407] Connection header received
0806 04:34:46.638337 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.638344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.638552 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.638730 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.639701 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.639707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.639719 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.639729 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.639768 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.639800 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.640143 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.640198 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.640275 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":115,"threads_started":1}
I20260806 04:34:46.640681 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.638263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:36639 (local address 127.0.0.1:53938)
0806 04:34:46.639113 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.639125 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.639196 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.639739 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.639742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.639756 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.640127 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.640129 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.640140 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.640169 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.640503 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.640506 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.640509 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.640563 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.640594 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":94,"threads_started":1}
I20260806 04:34:46.641068 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.638566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.12.15.194:36639)
0806 04:34:46.639443 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.639452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.639470 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:34:46.639511 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.639515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.639635 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.639728 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.640262 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.640267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.640277 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.640287 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.640324 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.640386 (+ 62us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.640855 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.640901 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.640960 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":115,"threads_started":1}
I20260806 04:34:46.643165 19506 tablet_service.cc:1467] Processing CreateTablet for tablet 72d55f6fbb7443339a3272eb034e932a (DEFAULT_TABLE table=test-table [id=e5325ec250f44e89a846f42a9ff7a80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:46.643292 19703 tablet_service.cc:1467] Processing CreateTablet for tablet 72d55f6fbb7443339a3272eb034e932a (DEFAULT_TABLE table=test-table [id=e5325ec250f44e89a846f42a9ff7a80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:46.644111 19703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d55f6fbb7443339a3272eb034e932a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:46.644114 19506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d55f6fbb7443339a3272eb034e932a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:46.645921 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.637476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:33379 (local address 127.0.0.1:50206)
0806 04:34:46.638939 (+ 1463us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.638953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.639028 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.644425 (+ 5397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.644428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.644443 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.645305 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.645307 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.645317 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.645343 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.645693 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.645696 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.645699 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.645775 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.645809 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"thread_start_us":105,"threads_started":1}
I20260806 04:34:46.646085 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.638209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.12.15.195:33379)
0806 04:34:46.641426 (+ 3217us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.641448 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.641471 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:34:46.641518 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.641523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.641680 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.644696 (+ 3016us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.645481 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.645488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.645500 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.645514 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.645559 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.645595 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.645707 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.645797 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.645863 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3116,"thread_start_us":151,"threads_started":1}
I20260806 04:34:46.648972 19896 tablet_service.cc:1467] Processing CreateTablet for tablet 72d55f6fbb7443339a3272eb034e932a (DEFAULT_TABLE table=test-table [id=e5325ec250f44e89a846f42a9ff7a80c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:46.650029 19896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d55f6fbb7443339a3272eb034e932a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:46.650614 20017 tablet_bootstrap.cc:492] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Bootstrap starting.
I20260806 04:34:46.651937 20020 tablet_bootstrap.cc:492] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: Bootstrap starting.
I20260806 04:34:46.653098 20017 tablet_bootstrap.cc:654] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:46.654003 20020 tablet_bootstrap.cc:654] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:46.653972 20017 log.cc:824] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:46.654740 20020 log.cc:824] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:46.655956 20017 tablet_bootstrap.cc:492] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: No bootstrap required, opened a new log
I20260806 04:34:46.656150 20017 ts_tablet_manager.cc:1397] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:34:46.656661 20020 tablet_bootstrap.cc:492] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: No bootstrap required, opened a new log
I20260806 04:34:46.656667 20023 tablet_bootstrap.cc:492] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Bootstrap starting.
I20260806 04:34:46.656840 20020 ts_tablet_manager.cc:1397] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:34:46.658463 20023 tablet_bootstrap.cc:654] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:46.659415 20023 log.cc:824] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:46.660596 20017 raft_consensus.cc:348] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.660866 20017 raft_consensus.cc:374] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:46.661000 20017 raft_consensus.cc:729] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b075cdcc0bf1480fa38606827a763a77, State: Initialized, Role: FOLLOWER
I20260806 04:34:46.661039 20023 tablet_bootstrap.cc:492] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: No bootstrap required, opened a new log
I20260806 04:34:46.661018 20020 raft_consensus.cc:348] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.661211 20023 ts_tablet_manager.cc:1397] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:34:46.661249 20020 raft_consensus.cc:374] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:46.661319 20020 raft_consensus.cc:729] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0d60b0c6e0d45d288c2c5e0f43bf5e6, State: Initialized, Role: FOLLOWER
I20260806 04:34:46.661531 20017 consensus_queue.cc:260] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.661707 20020 consensus_queue.cc:260] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.662235 19795 heartbeater.cc:503] Master 127.12.15.254:44657 was elected leader, sending a full tablet report...
I20260806 04:34:46.662591 20020 ts_tablet_manager.cc:1428] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:34:46.662935 19607 heartbeater.cc:503] Master 127.12.15.254:44657 was elected leader, sending a full tablet report...
I20260806 04:34:46.663579 20017 ts_tablet_manager.cc:1428] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260806 04:34:46.665278 20023 raft_consensus.cc:348] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.665518 20023 raft_consensus.cc:374] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:46.665583 20023 raft_consensus.cc:729] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2143cd71429d4188acf66b1ca67e7512, State: Initialized, Role: FOLLOWER
I20260806 04:34:46.666038 20023 consensus_queue.cc:260] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.666656 19992 heartbeater.cc:503] Master 127.12.15.254:44657 was elected leader, sending a full tablet report...
I20260806 04:34:46.666918 20023 ts_tablet_manager.cc:1428] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 04:34:46.684734 19797 tablet.cc:2369] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:46.755117 20026 raft_consensus.cc:482] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:46.755326 20026 raft_consensus.cc:504] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.756671 20026 leader_election.cc:288] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0d60b0c6e0d45d288c2c5e0f43bf5e6 (127.12.15.193:38849), 2143cd71429d4188acf66b1ca67e7512 (127.12.15.195:33379)
I20260806 04:34:46.758380 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.757049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:56515 (local address 127.12.15.193:38849)
0806 04:34:46.757174 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.757179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.757195 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:34:46.757242 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.757245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.757285 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.757350 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.758022 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.758025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.758027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.758039 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.758059 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.758067 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.758139 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.758224 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.758261 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:34:46.758380 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.756878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38849 (local address 127.12.15.194:56515)
0806 04:34:46.757004 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.757020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.757111 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.757389 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.757391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.757406 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.757864 (+ 458us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.757867 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.757880 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.757908 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.758135 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.758138 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.758141 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.758214 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.758252 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 04:34:46.758981 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.756878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:33379 (local address 127.12.15.194:51277)
0806 04:34:46.757719 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.757730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:46.757809 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:46.758039 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:46.758043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:46.758058 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:46.758447 (+ 389us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:34:46.758449 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:46.758462 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:34:46.758491 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:46.758766 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:46.758769 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:46.758773 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:34:46.758841 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 04:34:46.758870 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":522,"thread_start_us":97,"threads_started":1}
I20260806 04:34:46.759001 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.757124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:51277 (local address 127.12.15.195:33379)
0806 04:34:46.757282 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.757286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.757804 (+ 518us) server_negotiation.cc:407] Connection header received
0806 04:34:46.757890 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.757893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.757926 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.757996 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.758646 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.758649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.758652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.758665 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.758684 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.758693 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.758775 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.758842 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.758881 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 04:34:46.759016 19545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d55f6fbb7443339a3272eb034e932a" candidate_uuid: "b075cdcc0bf1480fa38606827a763a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" is_pre_election: true
I20260806 04:34:46.759294 19545 raft_consensus.cc:2454] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b075cdcc0bf1480fa38606827a763a77 in term 0.
I20260806 04:34:46.759582 19925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d55f6fbb7443339a3272eb034e932a" candidate_uuid: "b075cdcc0bf1480fa38606827a763a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2143cd71429d4188acf66b1ca67e7512" is_pre_election: true
I20260806 04:34:46.759768 19636 leader_election.cc:302] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b075cdcc0bf1480fa38606827a763a77, c0d60b0c6e0d45d288c2c5e0f43bf5e6; no voters:
I20260806 04:34:46.759878 19925 raft_consensus.cc:2454] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b075cdcc0bf1480fa38606827a763a77 in term 0.
I20260806 04:34:46.760056 20026 raft_consensus.cc:2781] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:46.760145 20026 raft_consensus.cc:482] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:46.760193 20026 raft_consensus.cc:3037] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:46.761246 20026 raft_consensus.cc:504] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
I20260806 04:34:46.761548 20026 leader_election.cc:288] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [CANDIDATE]: Term 1 election: Requested vote from peers c0d60b0c6e0d45d288c2c5e0f43bf5e6 (127.12.15.193:38849), 2143cd71429d4188acf66b1ca67e7512 (127.12.15.195:33379)
I20260806 04:34:46.761919 19545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d55f6fbb7443339a3272eb034e932a" candidate_uuid: "b075cdcc0bf1480fa38606827a763a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6"
I20260806 04:34:46.762009 19925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d55f6fbb7443339a3272eb034e932a" candidate_uuid: "b075cdcc0bf1480fa38606827a763a77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2143cd71429d4188acf66b1ca67e7512"
I20260806 04:34:46.762046 19545 raft_consensus.cc:3037] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:46.762130 19925 raft_consensus.cc:3037] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:46.763042 19545 raft_consensus.cc:2454] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b075cdcc0bf1480fa38606827a763a77 in term 1.
I20260806 04:34:46.763149 19925 raft_consensus.cc:2454] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b075cdcc0bf1480fa38606827a763a77 in term 1.
I20260806 04:34:46.763294 19636 leader_election.cc:302] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b075cdcc0bf1480fa38606827a763a77, c0d60b0c6e0d45d288c2c5e0f43bf5e6; no voters:
I20260806 04:34:46.763459 20026 raft_consensus.cc:2781] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:46.763672 20026 raft_consensus.cc:686] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 1 LEADER]: Becoming Leader. State: Replica: b075cdcc0bf1480fa38606827a763a77, State: Running, Role: LEADER
I20260806 04:34:46.763988 20026 consensus_queue.cc:237] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } }
W20260806 04:34:46.764497 19609 tablet.cc:2369] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:46.767079 19324 catalog_manager.cc:5495] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 reported cstate change: term changed from 0 to 1, leader changed from <none> to b075cdcc0bf1480fa38606827a763a77 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "b075cdcc0bf1480fa38606827a763a77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b075cdcc0bf1480fa38606827a763a77" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 36639 } health_report { overall_health: HEALTHY } } }
I20260806 04:34:46.792486 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.791175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.12.15.194:36639)
0806 04:34:46.791293 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:46.791298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:46.791315 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:34:46.791386 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:46.791389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:46.791443 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:46.791530 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:46.792147 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:46.792149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:46.792151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:46.792161 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:46.792177 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:46.792184 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:46.792252 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:46.792309 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 04:34:46.792363 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:34:46.798452 19545 raft_consensus.cc:1261] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 1 FOLLOWER]: Refusing update from remote peer b075cdcc0bf1480fa38606827a763a77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:46.798463 19925 raft_consensus.cc:1261] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 1 FOLLOWER]: Refusing update from remote peer b075cdcc0bf1480fa38606827a763a77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:46.799274 20026 consensus_queue.cc:1035] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0d60b0c6e0d45d288c2c5e0f43bf5e6" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:46.799594 20033 consensus_queue.cc:1035] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2143cd71429d4188acf66b1ca67e7512" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:46.809444 20036 mvcc.cc:205] Tried to move back new op lower bound from 7315418672318361600 to 7315418672188133376. Current Snapshot: MvccSnapshot[applied={T|T < 7315418672318361600 or (T in {7315418672318361600})}]
I20260806 04:34:46.809655 20037 mvcc.cc:205] Tried to move back new op lower bound from 7315418672318361600 to 7315418672188133376. Current Snapshot: MvccSnapshot[applied={T|T < 7315418672318361600}]
I20260806 04:34:46.816385 19324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:46.816545 19324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:46.818809 19324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 724faf9d039d486aa64439600a2ec69c: Sending DeleteTablet for 3 replicas of tablet 72d55f6fbb7443339a3272eb034e932a
I20260806 04:34:46.819538 19506 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d55f6fbb7443339a3272eb034e932a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:46 UTC) from {username='slave'} at 127.0.0.1:35118
I20260806 04:34:46.819608 19703 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d55f6fbb7443339a3272eb034e932a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:46 UTC) from {username='slave'} at 127.0.0.1:53938
I20260806 04:34:46.819591 19896 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d55f6fbb7443339a3272eb034e932a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:46 UTC) from {username='slave'} at 127.0.0.1:50206
I20260806 04:34:46.820211 20047 tablet_replica.cc:330] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6: stopping tablet replica
I20260806 04:34:46.820210 20051 tablet_replica.cc:330] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: stopping tablet replica
I20260806 04:34:46.820727 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 19419
I20260806 04:34:46.820725 20053 tablet_replica.cc:330] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: stopping tablet replica
I20260806 04:34:46.820950 20047 raft_consensus.cc:2229] T 72d55f6fbb7443339a3272eb034e932a P c0d60b0c6e0d45d288c2c5e0f43bf5e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:46.821013 20053 raft_consensus.cc:2229] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:46.831864 20053 raft_consensus.cc:2258] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:46.832125 20051 raft_consensus.cc:2229] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:46.832790 20051 raft_consensus.cc:2258] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:46.834458 20051 ts_tablet_manager.cc:1905] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:46.834481 20053 ts_tablet_manager.cc:1905] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:46.837273 20053 ts_tablet_manager.cc:1918] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:46.837280 20051 ts_tablet_manager.cc:1918] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:46.837371 20051 log.cc:1196] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/72d55f6fbb7443339a3272eb034e932a
I20260806 04:34:46.837491 20053 log.cc:1196] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/72d55f6fbb7443339a3272eb034e932a
I20260806 04:34:46.837718 20051 ts_tablet_manager.cc:1939] T 72d55f6fbb7443339a3272eb034e932a P b075cdcc0bf1480fa38606827a763a77: Deleting consensus metadata
I20260806 04:34:46.837941 20053 ts_tablet_manager.cc:1939] T 72d55f6fbb7443339a3272eb034e932a P 2143cd71429d4188acf66b1ca67e7512: Deleting consensus metadata
I20260806 04:34:46.838896 19306 catalog_manager.cc:4841] TS b075cdcc0bf1480fa38606827a763a77 (127.12.15.194:36639): tablet 72d55f6fbb7443339a3272eb034e932a (table test-table [id=e5325ec250f44e89a846f42a9ff7a80c]) successfully deleted
I20260806 04:34:46.839161 19305 catalog_manager.cc:4841] TS 2143cd71429d4188acf66b1ca67e7512 (127.12.15.195:33379): tablet 72d55f6fbb7443339a3272eb034e932a (table test-table [id=e5325ec250f44e89a846f42a9ff7a80c]) successfully deleted
W20260806 04:34:46.840031 19307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:38849 (error 108)
I20260806 04:34:46.840797 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 19611
I20260806 04:34:46.841089 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:46.840814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38849 (local address 127.0.0.1:35134)
0806 04:34:46.840936 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:46.841001 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 04:34:46.848309 19307 catalog_manager.cc:4584] TS c0d60b0c6e0d45d288c2c5e0f43bf5e6 (127.12.15.193:38849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72d55f6fbb7443339a3272eb034e932a: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38849: connect: Connection refused (error 111)
I20260806 04:34:46.853647 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 19799
I20260806 04:34:46.862180 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 19283
2026-08-06T04:34:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1107 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 04:34:46.888253 12351 mini_oidc.cc:152] Starting JWKS server
I20260806 04:34:46.890033 12351 webserver.cc:460] Webserver started at https://127.0.0.1:39435/ using document root <none> and password file <none>
I20260806 04:34:46.890184 12351 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 04:34:46.890406 12351 webserver.cc:460] Webserver started at http://127.0.0.1:46867/ using document root <none> and password file <none>
2026-08-06T04:34:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:46Z Disabled control of system clock
I20260806 04:34:46.899868 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40519
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:43759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:40519
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39435/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/testchainca.cert with env {}
W20260806 04:34:47.010667 20064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:47.010936 20064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:47.010990 20064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:47.015041 20064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:47.015101 20064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:47.015122 20064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:47.015144 20064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:47.015170 20064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:34:47.015201 20064 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:34:47.015221 20064 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39435/jwks.json
I20260806 04:34:47.020597 20064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:43759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:40519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39435/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:47.021701 20064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:47.022938 20064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:47.031320 20078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:47.031526 20076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:47.031648 20064 server_base.cc:1034] running on GCE node
W20260806 04:34:47.031345 20075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:47.032259 20064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:47.033397 20064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:47.034574 20064 hybrid_clock.cc:648] HybridClock initialized: now 1785990887034559 us; error 30 us; skew 500 ppm
I20260806 04:34:47.036434 20064 webserver.cc:460] Webserver started at http://127.12.15.254:38525/ using document root <none> and password file <none>
I20260806 04:34:47.037030 20064 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:47.037093 20064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:47.037292 20064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:47.039309 20064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "cd343d52e8c2437d906708b533016c12"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "3ea4fa3b6bbb61e0b2c3dda8f8cac3c0"
server_key_iv: "93146bf1085fd411b6b16e4c2b4fabd2"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.039856 20064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "cd343d52e8c2437d906708b533016c12"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "3ea4fa3b6bbb61e0b2c3dda8f8cac3c0"
server_key_iv: "93146bf1085fd411b6b16e4c2b4fabd2"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.043586 20064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 04:34:47.046001 20087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:47.047202 20064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:47.047309 20064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "cd343d52e8c2437d906708b533016c12"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "3ea4fa3b6bbb61e0b2c3dda8f8cac3c0"
server_key_iv: "93146bf1085fd411b6b16e4c2b4fabd2"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.047415 20064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:47.063781 20064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:47.064471 20064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:47.064628 20064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:47.071292 20064 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:40519
I20260806 04:34:47.071302 20177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:40519 every 8 connection(s)
I20260806 04:34:47.072284 20064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:47.075399 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 20064
I20260806 04:34:47.075538 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:47.075846 12351 external_mini_cluster.cc:1428] Setting key 148ed01141914bca98e9f782d2e0e9ea
I20260806 04:34:47.075874 20178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:47.082206 20178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12: Bootstrap starting.
I20260806 04:34:47.084508 20178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:47.084841 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:47.077336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.12.15.254:40519)
0806 04:34:47.077971 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:47.077985 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:47.078022 (+ 37us) server_negotiation.cc:407] Connection header received
0806 04:34:47.078643 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:47.078661 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:47.079053 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:47.079363 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:47.079808 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.080808 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.081684 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.082157 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.082306 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.083119 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:47.083124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:47.083137 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:47.083513 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:47.083566 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:47.083586 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:47.083671 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:47.083811 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 04:34:47.084062 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":126,"threads_started":1}
I20260806 04:34:47.085290 20178 log.cc:824] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:47.087466 20178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12: No bootstrap required, opened a new log
I20260806 04:34:47.087487 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:34:47.091243 20178 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd343d52e8c2437d906708b533016c12" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40519 } }
I20260806 04:34:47.091473 20178 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:47.091545 20178 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd343d52e8c2437d906708b533016c12, State: Initialized, Role: FOLLOWER
I20260806 04:34:47.091977 20178 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd343d52e8c2437d906708b533016c12" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40519 } }
I20260806 04:34:47.092110 20178 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:47.092177 20178 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:47.092274 20178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:47.093264 20178 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd343d52e8c2437d906708b533016c12" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40519 } }
I20260806 04:34:47.093605 20178 leader_election.cc:302] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd343d52e8c2437d906708b533016c12; no voters:
I20260806 04:34:47.093919 20178 leader_election.cc:288] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:47.094048 20187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:47.094274 20187 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [term 1 LEADER]: Becoming Leader. State: Replica: cd343d52e8c2437d906708b533016c12, State: Running, Role: LEADER
I20260806 04:34:47.094550 20187 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd343d52e8c2437d906708b533016c12" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40519 } }
I20260806 04:34:47.095110 20178 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:47.096010 20188 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd343d52e8c2437d906708b533016c12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd343d52e8c2437d906708b533016c12" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40519 } } }
I20260806 04:34:47.096148 20188 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:47.096516 20200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:47.096474 20190 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd343d52e8c2437d906708b533016c12. Latest consensus state: current_term: 1 leader_uuid: "cd343d52e8c2437d906708b533016c12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd343d52e8c2437d906708b533016c12" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40519 } } }
I20260806 04:34:47.096589 20190 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:47.099807 20200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:47.105336 20200 catalog_manager.cc:1347] Generated new cluster ID: 5153e2336065429398b3f0ab754b9272
I20260806 04:34:47.105412 20200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:47.130324 20200 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:47.132277 20200 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:47.142905 20200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd343d52e8c2437d906708b533016c12: Generated new TSK 0
I20260806 04:34:47.143754 20200 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 04:34:47.177244 20223 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 04:34:47.177429 20223 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 04:34:47.177984 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:47.160819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.12.15.254:40519)
0806 04:34:47.160964 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:47.160969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:47.161178 (+ 209us) server_negotiation.cc:407] Connection header received
0806 04:34:47.161402 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:47.161404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:47.161453 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:47.161537 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:34:47.161792 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.162316 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.163510 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.163730 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.163801 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.164159 (+ 358us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:34:47.177571 (+ 13412us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 04:34:47.177870 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44216: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 04:34:47.178042 20182 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44216: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 04:34:47.179615 20221 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:47.160720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40519 (local address 127.0.0.1:44216)
0806 04:34:47.161093 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:47.161109 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:47.161250 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:47.161564 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:47.161567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:47.161584 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:34:47.161699 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:47.161706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.162420 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.162424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:47.163397 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:47.163407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.163826 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.163830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:47.163919 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.163975 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:34:47.177864 (+ 13889us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 04:34:47.178138 (+ 274us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.254:40519: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.15.254:40519: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39435/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 04:34:47.180528 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 20064
2026-08-06T04:34:47Z chronyd exiting
I20260806 04:34:47.226336 12351 test_util.cc:175] -----------------------------------------------
I20260806 04:34:47.226426 12351 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785990859032933-12351-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (339 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 04:34:47.227550 12351 mini_oidc.cc:152] Starting JWKS server
I20260806 04:34:47.228408 12351 webserver.cc:460] Webserver started at https://127.0.0.1:38559/ using document root <none> and password file <none>
I20260806 04:34:47.228502 12351 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 04:34:47.228732 12351 webserver.cc:460] Webserver started at http://127.0.0.1:36837/ using document root <none> and password file <none>
2026-08-06T04:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:47Z Disabled control of system clock
I20260806 04:34:47.238492 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40273
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:40273
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38559/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/testchainca.cert with env {}
W20260806 04:34:47.350023 20230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:47.350275 20230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:47.350314 20230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:47.354172 20230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:47.354219 20230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:47.354233 20230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:47.354247 20230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:47.354260 20230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:34:47.354274 20230 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:34:47.354286 20230 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38559/jwks.json
I20260806 04:34:47.359551 20230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:40273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38559/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:47.360569 20230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:47.361515 20230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:47.369876 20237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:47.369982 20240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:47.370007 20242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:47.370136 20230 server_base.cc:1034] running on GCE node
I20260806 04:34:47.370705 20230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:47.371784 20230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:47.372956 20230 hybrid_clock.cc:648] HybridClock initialized: now 1785990887372939 us; error 28 us; skew 500 ppm
I20260806 04:34:47.374835 20230 webserver.cc:460] Webserver started at http://127.12.15.254:39759/ using document root <none> and password file <none>
I20260806 04:34:47.375391 20230 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:47.375458 20230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:47.375675 20230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:47.377736 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "e7ee1cb0b19044a8b2629a7c7787712b"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "c7630787d2d5fa900315f1908ec45bcc"
server_key_iv: "d07bc498cc4c49ade2abd14cdeac2513"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.378273 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "e7ee1cb0b19044a8b2629a7c7787712b"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "c7630787d2d5fa900315f1908ec45bcc"
server_key_iv: "d07bc498cc4c49ade2abd14cdeac2513"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.381877 20230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 04:34:47.384313 20256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:47.385548 20230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:34:47.385705 20230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "e7ee1cb0b19044a8b2629a7c7787712b"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "c7630787d2d5fa900315f1908ec45bcc"
server_key_iv: "d07bc498cc4c49ade2abd14cdeac2513"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.385807 20230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:47.412432 20230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:47.413184 20230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:47.413364 20230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:47.420286 20230 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:40273
I20260806 04:34:47.420303 20328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:40273 every 8 connection(s)
I20260806 04:34:47.421262 20230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:47.424311 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 20230
I20260806 04:34:47.424458 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:47.424721 20329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:47.424744 12351 external_mini_cluster.cc:1428] Setting key ed492dadf8ffd0ba293fdbbaa4ee71e6
I20260806 04:34:47.431133 20329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b: Bootstrap starting.
I20260806 04:34:47.433460 20329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:47.433729 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:47.426033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.12.15.254:40273)
0806 04:34:47.426715 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:47.426733 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:47.426768 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:34:47.427393 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:47.427410 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:47.427798 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:47.428097 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:47.428553 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.429601 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.430504 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.430979 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.431124 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.431949 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:47.431954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:47.431966 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:47.432377 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:47.432440 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:47.432462 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:47.432548 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:47.432686 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 04:34:47.432991 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":155,"threads_started":1}
I20260806 04:34:47.434333 20329 log.cc:824] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:47.436491 20329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b: No bootstrap required, opened a new log
I20260806 04:34:47.436694 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:34:47.440587 20329 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40273 } }
I20260806 04:34:47.440814 20329 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:47.440864 20329 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ee1cb0b19044a8b2629a7c7787712b, State: Initialized, Role: FOLLOWER
I20260806 04:34:47.441272 20329 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40273 } }
I20260806 04:34:47.441401 20329 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:47.441447 20329 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:47.441520 20329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:47.442365 20329 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40273 } }
I20260806 04:34:47.442690 20329 leader_election.cc:302] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7ee1cb0b19044a8b2629a7c7787712b; no voters:
I20260806 04:34:47.442952 20329 leader_election.cc:288] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:47.443090 20340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:47.443310 20340 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [term 1 LEADER]: Becoming Leader. State: Replica: e7ee1cb0b19044a8b2629a7c7787712b, State: Running, Role: LEADER
I20260806 04:34:47.443650 20340 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40273 } }
I20260806 04:34:47.444128 20329 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:47.445226 20341 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40273 } } }
I20260806 04:34:47.445266 20342 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7ee1cb0b19044a8b2629a7c7787712b. Latest consensus state: current_term: 1 leader_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ee1cb0b19044a8b2629a7c7787712b" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40273 } } }
I20260806 04:34:47.445358 20341 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:47.445407 20342 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:47.445758 20351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:47.448913 20351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:47.454505 20351 catalog_manager.cc:1347] Generated new cluster ID: 1b563d70602c418384ffa4cdc42e7dff
I20260806 04:34:47.454558 20351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:47.462950 20351 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:47.463845 20351 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:47.469316 20351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7ee1cb0b19044a8b2629a7c7787712b: Generated new TSK 0
I20260806 04:34:47.469969 20351 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 04:34:47.506537 20376 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 04:34:47.506645 20376 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 04:34:47.507465 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:47.485131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.12.15.254:40273)
0806 04:34:47.485278 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:47.485283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:47.485534 (+ 251us) server_negotiation.cc:407] Connection header received
0806 04:34:47.485791 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:47.485795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:47.485859 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:47.485968 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:34:47.486226 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.487025 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.488293 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.488650 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.488766 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.489130 (+ 364us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:34:47.507015 (+ 17885us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 04:34:47.507316 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 04:34:47.507604 20332 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 04:34:47.507745 20375 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:47.485145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40273 (local address 127.0.0.1:53692)
0806 04:34:47.485440 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:47.485456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:47.485623 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:47.485987 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:47.485989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:47.485996 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:34:47.486121 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:47.486128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.487154 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.487159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:47.488162 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:47.488172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.488781 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.488785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:47.488883 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.488914 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:34:47.507266 (+ 18352us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 04:34:47.507488 (+ 222us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.254:40273: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260806 04:34:47.508603 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 20230
2026-08-06T04:34:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (325 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 04:34:47.552997 12351 mini_oidc.cc:152] Starting JWKS server
I20260806 04:34:47.553849 12351 webserver.cc:460] Webserver started at https://127.0.0.1:40531/ using document root <none> and password file <none>
I20260806 04:34:47.553946 12351 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 04:34:47.554189 12351 webserver.cc:460] Webserver started at http://127.0.0.1:40009/ using document root <none> and password file <none>
2026-08-06T04:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:47Z Disabled control of system clock
I20260806 04:34:47.564049 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38507
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:38507
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40531/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 04:34:47.674816 20385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:47.675102 20385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:47.675143 20385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:47.679215 20385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:47.679279 20385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:47.679293 20385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:47.679309 20385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:47.679325 20385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:34:47.679355 20385 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:34:47.679381 20385 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40531/jwks.json
I20260806 04:34:47.684301 20385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:38507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40531/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:47.685583 20385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:47.687150 20385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:47.695926 20392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:47.695955 20396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:47.695955 20393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:47.696305 20385 server_base.cc:1034] running on GCE node
I20260806 04:34:47.696868 20385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:47.697947 20385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:47.699131 20385 hybrid_clock.cc:648] HybridClock initialized: now 1785990887699114 us; error 33 us; skew 500 ppm
I20260806 04:34:47.701032 20385 webserver.cc:460] Webserver started at http://127.12.15.254:43757/ using document root <none> and password file <none>
I20260806 04:34:47.701610 20385 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:47.701681 20385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:47.701890 20385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:47.703981 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "7953eb89339a40bb890751b88470a06e"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "f6ce232e4d8bc03c5e4b2f6401d66a1f"
server_key_iv: "9413b9a017da1ed2a71d0e8741e163b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.704576 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "7953eb89339a40bb890751b88470a06e"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "f6ce232e4d8bc03c5e4b2f6401d66a1f"
server_key_iv: "9413b9a017da1ed2a71d0e8741e163b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.708719 20385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:34:47.711313 20406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:47.712728 20385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:47.712853 20385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "7953eb89339a40bb890751b88470a06e"
format_stamp: "Formatted at 2026-08-06 04:34:47 on dist-test-slave-xt4k"
server_key: "f6ce232e4d8bc03c5e4b2f6401d66a1f"
server_key_iv: "9413b9a017da1ed2a71d0e8741e163b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:47.712960 20385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:47.737644 20385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:47.738360 20385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:47.738539 20385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:47.745374 20385 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:38507
I20260806 04:34:47.745386 20484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:38507 every 8 connection(s)
I20260806 04:34:47.746306 20385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:47.749485 20485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:47.750025 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 20385
I20260806 04:34:47.750145 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:47.750409 12351 external_mini_cluster.cc:1428] Setting key dce4090467a1ea167461054e2bfc4035
I20260806 04:34:47.756319 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e: Bootstrap starting.
I20260806 04:34:47.759027 20485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:47.759972 20485 log.cc:824] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:47.760850 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:47.751700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.12.15.254:38507)
0806 04:34:47.752404 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:47.752425 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:47.752473 (+ 48us) server_negotiation.cc:407] Connection header received
0806 04:34:47.753289 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:47.753315 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:47.753836 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:47.754207 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:47.754708 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.756135 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.757117 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.757803 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.757963 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.758737 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:47.758745 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:47.758762 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:47.759289 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:47.759365 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:47.759393 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:47.759491 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:47.759637 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 04:34:47.759932 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":102,"threads_started":1}
I20260806 04:34:47.762207 20485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e: No bootstrap required, opened a new log
I20260806 04:34:47.763764 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:34:47.766259 20485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7953eb89339a40bb890751b88470a06e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38507 } }
I20260806 04:34:47.766497 20485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:47.766531 20485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7953eb89339a40bb890751b88470a06e, State: Initialized, Role: FOLLOWER
I20260806 04:34:47.766945 20485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7953eb89339a40bb890751b88470a06e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38507 } }
I20260806 04:34:47.767086 20485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:47.767143 20485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:47.767220 20485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:47.768167 20485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7953eb89339a40bb890751b88470a06e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38507 } }
I20260806 04:34:47.768518 20485 leader_election.cc:302] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7953eb89339a40bb890751b88470a06e; no voters:
I20260806 04:34:47.768868 20485 leader_election.cc:288] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:47.769055 20490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:47.769290 20490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [term 1 LEADER]: Becoming Leader. State: Replica: 7953eb89339a40bb890751b88470a06e, State: Running, Role: LEADER
I20260806 04:34:47.769600 20490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7953eb89339a40bb890751b88470a06e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38507 } }
I20260806 04:34:47.770176 20485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:47.771835 20492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7953eb89339a40bb890751b88470a06e. Latest consensus state: current_term: 1 leader_uuid: "7953eb89339a40bb890751b88470a06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7953eb89339a40bb890751b88470a06e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38507 } } }
I20260806 04:34:47.771982 20492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:47.772215 20491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7953eb89339a40bb890751b88470a06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7953eb89339a40bb890751b88470a06e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38507 } } }
I20260806 04:34:47.772426 20491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e [sys.catalog]: This master's current role is: LEADER
W20260806 04:34:47.775103 20508 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:34:47.775172 20508 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:34:47.775254 20507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:47.775965 20507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:47.781457 20507 catalog_manager.cc:1347] Generated new cluster ID: 335597437bff44d98b1d1333d1d65aeb
I20260806 04:34:47.781526 20507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:47.796723 20507 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:47.797974 20507 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:47.804878 20507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7953eb89339a40bb890751b88470a06e: Generated new TSK 0
I20260806 04:34:47.805675 20507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:34:47.823021 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:47.818252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58972 (local address 127.12.15.254:38507)
0806 04:34:47.818393 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:47.818398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:47.818561 (+ 163us) server_negotiation.cc:407] Connection header received
0806 04:34:47.818718 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:47.818720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:47.818771 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:47.818860 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:34:47.819082 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.819637 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.820863 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.821079 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.821154 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.821439 (+ 285us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:34:47.822631 (+ 1192us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 04:34:47.822859 (+ 228us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 04:34:47.823096 20525 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:47.818168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38507 (local address 127.0.0.1:58972)
0806 04:34:47.818471 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:47.818486 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:47.818603 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:47.818871 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:47.818874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:47.818882 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:34:47.818992 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:47.818998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.819734 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.819737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:47.820758 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:47.820770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:47.821177 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:47.821181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:47.821270 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:47.821298 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:34:47.822782 (+ 1484us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 04:34:47.822919 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.254:38507: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":91,"threads_started":1}
W20260806 04:34:47.823103 20488 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260806 04:34:47.823915 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 20385
E20260806 04:34:47.839226 20526 webserver.cc:533] Webserver: sslize failed:
E20260806 04:34:47.839306 20526 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T04:34:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (316 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T04:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:47Z Disabled control of system clock
I20260806 04:34:47.878716 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33383
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:33383
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/testchainca.cert with env {}
W20260806 04:34:47.990528 20530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:47.990816 20530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:47.990856 20530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:47.994866 20530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:47.994932 20530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:47.994947 20530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:47.994963 20530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:47.994976 20530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 04:34:47.994992 20530 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 04:34:47.995003 20530 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 04:34:48.000481 20530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:33383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:48.001555 20530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:48.002543 20530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:48.011049 20537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:48.011103 20530 server_base.cc:1034] running on GCE node
W20260806 04:34:48.011044 20539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:48.011057 20541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:48.011868 20530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:48.013082 20530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:48.014247 20530 hybrid_clock.cc:648] HybridClock initialized: now 1785990888014232 us; error 27 us; skew 500 ppm
I20260806 04:34:48.016084 20530 webserver.cc:460] Webserver started at http://127.12.15.254:42557/ using document root <none> and password file <none>
I20260806 04:34:48.016700 20530 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:48.016772 20530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:48.016978 20530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:48.019034 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "b010292eebec41ce959a9c495edce0d8"
format_stamp: "Formatted at 2026-08-06 04:34:48 on dist-test-slave-xt4k"
server_key: "62c31b601bdf16f01c11f1f8a0b025ac"
server_key_iv: "d45031587b0b70c256175f4b040080cd"
server_key_version: "encryptionkey@0"
I20260806 04:34:48.019589 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "b010292eebec41ce959a9c495edce0d8"
format_stamp: "Formatted at 2026-08-06 04:34:48 on dist-test-slave-xt4k"
server_key: "62c31b601bdf16f01c11f1f8a0b025ac"
server_key_iv: "d45031587b0b70c256175f4b040080cd"
server_key_version: "encryptionkey@0"
I20260806 04:34:48.023268 20530 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:34:48.025766 20549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:48.026943 20530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:48.027068 20530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "b010292eebec41ce959a9c495edce0d8"
format_stamp: "Formatted at 2026-08-06 04:34:48 on dist-test-slave-xt4k"
server_key: "62c31b601bdf16f01c11f1f8a0b025ac"
server_key_iv: "d45031587b0b70c256175f4b040080cd"
server_key_version: "encryptionkey@0"
I20260806 04:34:48.027176 20530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:48.045718 20530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:48.046458 20530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:48.046638 20530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:48.053273 20530 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:33383
I20260806 04:34:48.053292 20631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:33383 every 8 connection(s)
I20260806 04:34:48.054210 20530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:48.054558 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 20530
I20260806 04:34:48.054688 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:48.054973 12351 external_mini_cluster.cc:1428] Setting key 48e9314a31f53cda363bdbd28a9a0f86
I20260806 04:34:48.057317 20632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:48.063589 20632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8: Bootstrap starting.
I20260806 04:34:48.064215 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:48.056257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.12.15.254:33383)
0806 04:34:48.056892 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:48.056908 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:48.056943 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:34:48.057614 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:48.057631 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:48.058018 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:48.058313 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:48.058648 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:48.059612 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:48.060796 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:48.061255 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:48.061369 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:48.062285 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:48.062291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:48.062304 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:34:48.062724 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:48.062789 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:34:48.062812 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:48.062896 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:48.063121 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 04:34:48.063412 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":117,"threads_started":1}
I20260806 04:34:48.066031 20632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:48.066784 20632 log.cc:824] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:48.067034 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:34:48.068854 20632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8: No bootstrap required, opened a new log
I20260806 04:34:48.072240 20632 raft_consensus.cc:348] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010292eebec41ce959a9c495edce0d8" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33383 } }
I20260806 04:34:48.072499 20632 raft_consensus.cc:374] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:48.072559 20632 raft_consensus.cc:729] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b010292eebec41ce959a9c495edce0d8, State: Initialized, Role: FOLLOWER
I20260806 04:34:48.072989 20632 consensus_queue.cc:260] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010292eebec41ce959a9c495edce0d8" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33383 } }
I20260806 04:34:48.073114 20632 raft_consensus.cc:388] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:48.073170 20632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:48.073267 20632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:48.074179 20632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010292eebec41ce959a9c495edce0d8" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33383 } }
I20260806 04:34:48.074532 20632 leader_election.cc:302] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b010292eebec41ce959a9c495edce0d8; no voters:
I20260806 04:34:48.074872 20632 leader_election.cc:288] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:48.074963 20640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:48.075191 20640 raft_consensus.cc:686] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [term 1 LEADER]: Becoming Leader. State: Replica: b010292eebec41ce959a9c495edce0d8, State: Running, Role: LEADER
I20260806 04:34:48.075500 20640 consensus_queue.cc:237] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010292eebec41ce959a9c495edce0d8" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33383 } }
I20260806 04:34:48.076014 20632 sys_catalog.cc:564] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:48.077163 20644 sys_catalog.cc:455] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b010292eebec41ce959a9c495edce0d8. Latest consensus state: current_term: 1 leader_uuid: "b010292eebec41ce959a9c495edce0d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010292eebec41ce959a9c495edce0d8" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33383 } } }
I20260806 04:34:48.077301 20644 sys_catalog.cc:458] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:48.077481 20643 sys_catalog.cc:455] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b010292eebec41ce959a9c495edce0d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010292eebec41ce959a9c495edce0d8" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33383 } } }
I20260806 04:34:48.077668 20643 sys_catalog.cc:458] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:48.077730 20654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:48.080874 20654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:48.086344 20654 catalog_manager.cc:1347] Generated new cluster ID: beb4284c55834a3f847eba731c43fe84
I20260806 04:34:48.086421 20654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:48.101634 20654 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:48.102550 20654 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:48.110975 20654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b010292eebec41ce959a9c495edce0d8: Generated new TSK 0
I20260806 04:34:48.111641 20654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:34:48.156266 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:48.121023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.12.15.254:33383)
0806 04:34:48.121147 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:48.121151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:48.121297 (+ 146us) server_negotiation.cc:407] Connection header received
0806 04:34:48.121435 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:48.121438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:48.121492 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:48.121600 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0806 04:34:48.121799 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:48.122681 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:48.123478 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:48.123812 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:48.123906 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:48.124187 (+ 281us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 04:34:48.155736 (+ 31549us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 04:34:48.156104 (+ 368us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44378: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 04:34:48.156387 20638 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44378: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 04:34:48.156378 20675 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:48.120932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33383 (local address 127.0.0.1:44378)
0806 04:34:48.121213 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:48.121226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:48.121322 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:48.121614 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:48.121617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:48.121622 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0806 04:34:48.121713 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:48.121718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:48.122774 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:48.122777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:48.123378 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:48.123386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:48.123938 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:48.123941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:48.124009 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:48.124032 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 04:34:48.155996 (+ 31964us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 04:34:48.156186 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.254:33383: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260806 04:34:48.157090 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 20530
2026-08-06T04:34:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (313 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:48 dist-test-slave-xt4k krb5kdc[20691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:48 dist-test-slave-xt4k krb5kdc[20691](info): set up 2 sockets
Aug 06 04:34:48 dist-test-slave-xt4k krb5kdc[20691](info): commencing operation
krb5kdc: starting...
W20260806 04:34:49.562873 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:49 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:34:49.683552 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40397
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:49.795600 20716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:49.795879 20716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:49.795919 20716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:49.799957 20716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:49.800026 20716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:49.800045 20716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:49.800060 20716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:49.800073 20716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:49.805472 20716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:49.806560 20716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:49.807576 20716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:49.816447 20728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:49.816486 20725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:49.816457 20726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:49.816676 20716 server_base.cc:1034] running on GCE node
I20260806 04:34:49.817242 20716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:49.818383 20716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:49.819569 20716 hybrid_clock.cc:648] HybridClock initialized: now 1785990889819550 us; error 30 us; skew 500 ppm
Aug 06 04:34:49 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990889, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:49.822417 20716 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:34:49.823307 20716 webserver.cc:460] Webserver started at http://127.12.15.254:35379/ using document root <none> and password file <none>
I20260806 04:34:49.823845 20716 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:49.823911 20716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:49.824129 20716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:49.826220 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "2319d338dc2641d089b375ea02a132ba"
format_stamp: "Formatted at 2026-08-06 04:34:49 on dist-test-slave-xt4k"
server_key: "5325d9c9b36722d3945e4557989f99d2"
server_key_iv: "bcf3d8dbac7b7d8b4458a592850321b3"
server_key_version: "encryptionkey@0"
I20260806 04:34:49.826803 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "2319d338dc2641d089b375ea02a132ba"
format_stamp: "Formatted at 2026-08-06 04:34:49 on dist-test-slave-xt4k"
server_key: "5325d9c9b36722d3945e4557989f99d2"
server_key_iv: "bcf3d8dbac7b7d8b4458a592850321b3"
server_key_version: "encryptionkey@0"
I20260806 04:34:49.830508 20716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 04:34:49.832906 20738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:49.834162 20716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:49.834295 20716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "2319d338dc2641d089b375ea02a132ba"
format_stamp: "Formatted at 2026-08-06 04:34:49 on dist-test-slave-xt4k"
server_key: "5325d9c9b36722d3945e4557989f99d2"
server_key_iv: "bcf3d8dbac7b7d8b4458a592850321b3"
server_key_version: "encryptionkey@0"
I20260806 04:34:49.834394 20716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:49.846627 20716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:49.849776 20716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:49.849961 20716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:49.857012 20716 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:40397
I20260806 04:34:49.857034 20815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:40397 every 8 connection(s)
I20260806 04:34:49.857976 20716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:49.859911 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 20716
I20260806 04:34:49.860050 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:49.860368 12351 external_mini_cluster.cc:1428] Setting key 790ff3e3994d08f9be746f7db2b5b3f8
I20260806 04:34:49.861083 20816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:49.866910 20816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba: Bootstrap starting.
Aug 06 04:34:49 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:49.869204 20816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:49.869908 20816 log.cc:824] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:49.871896 20816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba: No bootstrap required, opened a new log
I20260806 04:34:49.873778 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:49.861823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.12.15.254:40397)
0806 04:34:49.862271 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:49.862286 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:49.862322 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:34:49.862987 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:49.863005 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:49.863327 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:49.863624 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:49.864441 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:49.865430 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:49.866243 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:49.866702 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:49.866856 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:49.869210 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:49.869216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:49.869230 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:49.869258 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:49.871044 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:49.871463 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:49.871470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:49.871476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:49.871589 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:49.871846 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:49.871850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:49.871852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:49.872280 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:49.872492 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:49.872785 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 04:34:49.873080 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":143,"threads_started":1}
I20260806 04:34:49.875495 20816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2319d338dc2641d089b375ea02a132ba" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40397 } }
I20260806 04:34:49.875713 20816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:49.875800 20816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2319d338dc2641d089b375ea02a132ba, State: Initialized, Role: FOLLOWER
I20260806 04:34:49.876216 20816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2319d338dc2641d089b375ea02a132ba" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40397 } }
I20260806 04:34:49.876362 20816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:49.876428 20816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:49.876526 20816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:49.877408 20816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2319d338dc2641d089b375ea02a132ba" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40397 } }
I20260806 04:34:49.877731 20816 leader_election.cc:302] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2319d338dc2641d089b375ea02a132ba; no voters:
I20260806 04:34:49.878021 20816 leader_election.cc:288] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:49.878149 20821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:49.878386 20821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [term 1 LEADER]: Becoming Leader. State: Replica: 2319d338dc2641d089b375ea02a132ba, State: Running, Role: LEADER
I20260806 04:34:49.878700 20821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2319d338dc2641d089b375ea02a132ba" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40397 } }
I20260806 04:34:49.879150 20816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:49.880254 20824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2319d338dc2641d089b375ea02a132ba. Latest consensus state: current_term: 1 leader_uuid: "2319d338dc2641d089b375ea02a132ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2319d338dc2641d089b375ea02a132ba" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40397 } } }
I20260806 04:34:49.880390 20824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:49.880672 20823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2319d338dc2641d089b375ea02a132ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2319d338dc2641d089b375ea02a132ba" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40397 } } }
I20260806 04:34:49.880779 20833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:49.880825 20823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:49.884013 20833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:49.890463 20833 catalog_manager.cc:1347] Generated new cluster ID: 54dc7902c8784c02aca9618433f8be42
I20260806 04:34:49.890561 20833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:49.922870 20833 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:49.923921 20833 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:49.930612 20833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2319d338dc2641d089b375ea02a132ba: Generated new TSK 0
I20260806 04:34:49.931354 20833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:34:49.971817 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40397
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:50.100059 20855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:50.100297 20855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:50.100370 20855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:50.104321 20855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:50.104388 20855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:50.104442 20855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:50.109508 20855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:50.110433 20855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:50.111356 20855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:50.120875 20869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:50.120860 20863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:50.120879 20866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:50.121140 20855 server_base.cc:1034] running on GCE node
I20260806 04:34:50.121716 20855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:50.122359 20855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:50.123540 20855 hybrid_clock.cc:648] HybridClock initialized: now 1785990890123512 us; error 43 us; skew 500 ppm
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.126327 20855 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:34:50.127197 20855 webserver.cc:460] Webserver started at http://127.12.15.193:38735/ using document root <none> and password file <none>
I20260806 04:34:50.127774 20855 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:50.127842 20855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:50.128047 20855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:50.130092 20855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "4cf0a9a5ee7244aa8976e33c56f5e212"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "bc5454d262f02ef12af838569e1aa101"
server_key_iv: "fdd589c6b59eec1bb62a72b822bf45c4"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.130630 20855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "4cf0a9a5ee7244aa8976e33c56f5e212"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "bc5454d262f02ef12af838569e1aa101"
server_key_iv: "fdd589c6b59eec1bb62a72b822bf45c4"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.134302 20855 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:34:50.136891 20878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.138233 20855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:50.138361 20855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "4cf0a9a5ee7244aa8976e33c56f5e212"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "bc5454d262f02ef12af838569e1aa101"
server_key_iv: "fdd589c6b59eec1bb62a72b822bf45c4"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.138461 20855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:50.160576 20855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:50.163640 20855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:50.163828 20855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:50.164458 20855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:50.165310 20855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:50.165360 20855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.165423 20855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:50.165448 20855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.173867 20855 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:34097
I20260806 04:34:50.173888 21046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:34097 every 8 connection(s)
I20260806 04:34:50.174741 20855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:50.180014 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 20855
I20260806 04:34:50.180137 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:50.180472 12351 external_mini_cluster.cc:1428] Setting key 967e7ef848da04db00d2127cb4308b2b
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:50.188493 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.176460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:34277 (local address 127.12.15.254:40397)
0806 04:34:50.176629 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.176634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.177464 (+ 830us) server_negotiation.cc:407] Connection header received
0806 04:34:50.178799 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.178802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.178849 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.178911 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.180139 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.180929 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.182208 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.182452 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.182538 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.184834 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:50.184837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:50.184839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:50.184861 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:50.186187 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.186804 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.186808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.186810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.186893 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.187256 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.187258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.187259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.187441 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:50.187524 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:50.188203 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 04:34:50.188322 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:34:50.189640 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.176845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40397 (local address 127.12.15.193:34277)
0806 04:34:50.177316 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:50.177358 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:50.178573 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:50.179122 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:50.179129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:50.179467 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:50.179986 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:50.180001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.181056 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.181059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:50.182095 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:50.182106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.182588 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.182593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:50.182746 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.183278 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:50.183295 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:50.184678 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:50.186305 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:50.186312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:50.186329 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:50.186690 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:50.186975 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:50.186978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:50.186980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:50.187166 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:50.187551 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:50.187558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:50.187902 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 04:34:50.188143 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 04:34:50.188547 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":116,"threads_started":1}
I20260806 04:34:50.190949 21048 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40397
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:50.191210 21048 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:50.191761 21048 heartbeater.cc:511] Master 127.12.15.254:40397 requested a full tablet report, sending...
I20260806 04:34:50.193462 20764 ts_manager.cc:194] Registered new tserver with Master: 4cf0a9a5ee7244aa8976e33c56f5e212 (127.12.15.193:34097)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:34:50.194628 20764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:34277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:34:50.211238 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40397
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:50.326784 21055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:50.327056 21055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:50.327100 21055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:50.331139 21055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:50.331199 21055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:50.331256 21055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:50.336550 21055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:50.337579 21055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:50.338590 21055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:50.348312 21065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:50.348285 21063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:50.348302 21062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:50.348598 21055 server_base.cc:1034] running on GCE node
I20260806 04:34:50.349014 21055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:50.349614 21055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:50.350813 21055 hybrid_clock.cc:648] HybridClock initialized: now 1785990890350785 us; error 39 us; skew 500 ppm
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.353713 21055 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:34:50.354621 21055 webserver.cc:460] Webserver started at http://127.12.15.194:42231/ using document root <none> and password file <none>
I20260806 04:34:50.355175 21055 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:50.355242 21055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:50.355448 21055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:50.357538 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "9f9565c5a6ff48f6a4a8f413281266d9"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "043ad7c0866f5793cea7ebcbfab0440c"
server_key_iv: "ec629cdb46eb1d43dffa8ecc689eb349"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.358091 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "9f9565c5a6ff48f6a4a8f413281266d9"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "043ad7c0866f5793cea7ebcbfab0440c"
server_key_iv: "ec629cdb46eb1d43dffa8ecc689eb349"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.361738 21055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:34:50.364228 21075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.365517 21055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:50.365633 21055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "9f9565c5a6ff48f6a4a8f413281266d9"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "043ad7c0866f5793cea7ebcbfab0440c"
server_key_iv: "ec629cdb46eb1d43dffa8ecc689eb349"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.365729 21055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:50.380223 21055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:50.383389 21055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:50.383570 21055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:50.384167 21055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:50.385142 21055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:50.385191 21055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.385247 21055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:50.385272 21055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.393810 21055 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:34227
I20260806 04:34:50.393890 21255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:34227 every 8 connection(s)
I20260806 04:34:50.394774 21055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:50.399544 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 21055
I20260806 04:34:50.399669 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:50.399930 12351 external_mini_cluster.cc:1428] Setting key 2e10fdeaac457db9e48dc1e1d09a6e26
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:50.409121 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.396713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:47845 (local address 127.12.15.254:40397)
0806 04:34:50.396842 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.396846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.397694 (+ 848us) server_negotiation.cc:407] Connection header received
0806 04:34:50.399150 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.399154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.399215 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.399297 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.400832 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.401439 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.402598 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.402824 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.402914 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.405319 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:50.405323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:50.405326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:50.405352 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:50.406949 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.407512 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.407515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.407516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.407608 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.407929 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.407932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.407934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.408135 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:50.408221 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:50.408802 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 04:34:50.408940 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:50.409966 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.397065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40397 (local address 127.12.15.194:47845)
0806 04:34:50.397535 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:50.397580 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:50.398871 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:50.399477 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:50.399484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:50.400021 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:50.400636 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:50.400653 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.401564 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.401567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:50.402479 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:50.402489 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.402962 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.402967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:50.403134 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.403687 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:50.403703 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:50.405162 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:50.407079 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:50.407086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:50.407103 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:50.407378 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:50.407697 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:50.407700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:50.407702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:50.407840 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:50.408242 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:50.408249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:50.408553 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 04:34:50.408743 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 04:34:50.409081 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":114,"threads_started":1}
I20260806 04:34:50.411160 21257 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40397
I20260806 04:34:50.411486 21257 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:50.412041 21257 heartbeater.cc:511] Master 127.12.15.254:40397 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:34:50.413411 20764 ts_manager.cc:194] Registered new tserver with Master: 9f9565c5a6ff48f6a4a8f413281266d9 (127.12.15.194:34227)
I20260806 04:34:50.414016 20764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:47845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:34:50.431432 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40397
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:50.548318 21265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:50.548605 21265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:50.548666 21265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:50.552613 21265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:50.552670 21265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:50.552749 21265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:50.557924 21265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:50.558962 21265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:50.559958 21265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:50.569442 21273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:50.569533 21274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:50.569684 21265 server_base.cc:1034] running on GCE node
W20260806 04:34:50.569661 21276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:50.570230 21265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:50.570837 21265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:50.571995 21265 hybrid_clock.cc:648] HybridClock initialized: now 1785990890571981 us; error 26 us; skew 500 ppm
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.574826 21265 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:34:50.575665 21265 webserver.cc:460] Webserver started at http://127.12.15.195:32771/ using document root <none> and password file <none>
I20260806 04:34:50.576203 21265 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:50.576270 21265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:50.576515 21265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:50.578642 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "c5be16667e1d41369f25a7431b6021eb"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "dad35f35f480cfe9089fcae4830f62d5"
server_key_iv: "8b117be6436667ba893fadfb4252ac50"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.579213 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "c5be16667e1d41369f25a7431b6021eb"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "dad35f35f480cfe9089fcae4830f62d5"
server_key_iv: "8b117be6436667ba893fadfb4252ac50"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.583062 21265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:34:50.585477 21285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.586671 21265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:50.586784 21265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "c5be16667e1d41369f25a7431b6021eb"
format_stamp: "Formatted at 2026-08-06 04:34:50 on dist-test-slave-xt4k"
server_key: "dad35f35f480cfe9089fcae4830f62d5"
server_key_iv: "8b117be6436667ba893fadfb4252ac50"
server_key_version: "encryptionkey@0"
I20260806 04:34:50.586879 21265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:50.602665 21265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:50.605526 21265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:50.605679 21265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:50.606225 21265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:50.607018 21265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:50.607070 21265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.607110 21265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:50.607131 21265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:50.615453 21265 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:43745
I20260806 04:34:50.615492 21460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:43745 every 8 connection(s)
I20260806 04:34:50.616313 21265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:50.618932 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 21265
I20260806 04:34:50.619065 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:50.619371 12351 external_mini_cluster.cc:1428] Setting key f0f9751fdeaae5c322b5e0cea92548ff
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:34:50.629698 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.618107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:44585 (local address 127.12.15.254:40397)
0806 04:34:50.618246 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.618251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.619153 (+ 902us) server_negotiation.cc:407] Connection header received
0806 04:34:50.620825 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.620828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.620885 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.620985 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.622272 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.622819 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.623911 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.624134 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.624218 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.626527 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:50.626530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:50.626532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:50.626556 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:50.627808 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.628295 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.628298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.628299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.628418 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.628768 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.628771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.628772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.628956 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:50.629035 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:50.629475 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 04:34:50.629575 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:34:50.630522 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.618488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40397 (local address 127.12.15.195:44585)
0806 04:34:50.619002 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:50.619062 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:50.620596 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:50.621158 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:50.621165 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:50.621629 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:50.622144 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:50.622159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.622938 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.622940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:50.623800 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:50.623809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.624246 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.624249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:50.624442 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.625000 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:50.625016 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:50.626381 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:50.627919 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:50.627925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:50.627938 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:50.628190 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:50.628525 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:50.628527 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:50.628529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:50.628671 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:50.629038 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:50.629043 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:50.629284 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 04:34:50.629442 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 04:34:50.629690 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260806 04:34:50.631595 21461 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40397
I20260806 04:34:50.631832 21461 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:50.632443 21461 heartbeater.cc:511] Master 127.12.15.254:40397 requested a full tablet report, sending...
I20260806 04:34:50.633531 20764 ts_manager.cc:194] Registered new tserver with Master: c5be16667e1d41369f25a7431b6021eb (127.12.15.195:43745)
I20260806 04:34:50.634080 20764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:44585
I20260806 04:34:50.645315 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/test-admin.keytab.
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.660749 12351 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 04:34:50.663615 21474 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:50.662102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.12.15.193:34097)
0806 04:34:50.662435 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.662452 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.662475 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:34:50.662524 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.662529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.662717 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.662850 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.663447 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34240: BlockingRecv error: recv got EOF from 127.0.0.1:34240 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":99,"threads_started":1}
W20260806 04:34:50.663740 12351 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc' of type FILE
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.664777 12351 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.665643 12351 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:50.675248 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.666481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.12.15.193:34097)
0806 04:34:50.666624 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.666628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.666686 (+ 58us) server_negotiation.cc:407] Connection header received
0806 04:34:50.666817 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.666821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.666863 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.666929 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.667468 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.668249 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.669136 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.669514 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.669623 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.671899 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:50.671910 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:50.671925 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:50.671952 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:50.673559 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.674012 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.674019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.674020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.674104 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.674375 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.674377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.674378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.674641 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:50.674795 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:50.674984 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 04:34:50.675117 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 04:34:50.679035 21474 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:50.678185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.12.15.193:34097)
0806 04:34:50.678318 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.678321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.678381 (+ 60us) server_negotiation.cc:407] Connection header received
0806 04:34:50.678467 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.678470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.678511 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.678573 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.678952 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34248: BlockingRecv error: recv got EOF from 127.0.0.1:34248 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 04:34:50.679143 12351 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc' of type FILE
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.680164 12351 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.681228 12351 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:50.689280 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.682024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.12.15.193:34097)
0806 04:34:50.682132 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.682136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.682245 (+ 109us) server_negotiation.cc:407] Connection header received
0806 04:34:50.682325 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.682327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.682361 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.682439 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.682857 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.683357 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.684047 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.684264 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.684362 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.686356 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:50.686359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:50.686361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:50.686382 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:50.687636 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.688016 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.688020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.688021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.688103 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.688492 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.688495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.688496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.688691 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:50.688805 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:50.689069 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 04:34:50.689167 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 04:34:50.692173 21474 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:50.691298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.12.15.193:34097)
0806 04:34:50.691410 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.691414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.691577 (+ 163us) server_negotiation.cc:407] Connection header received
0806 04:34:50.691660 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.691662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.691697 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.691757 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.692073 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34264: BlockingRecv error: recv got EOF from 127.0.0.1:34264 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 04:34:50.692283 12351 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785990859032933-12351-0/krb5kdc/krb5cc' of type FILE
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.693255 12351 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:50.694031 12351 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[20691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:34:50.702248 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:50.694773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.12.15.193:34097)
0806 04:34:50.694882 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:50.694885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:50.694962 (+ 77us) server_negotiation.cc:407] Connection header received
0806 04:34:50.695051 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:50.695053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:50.695089 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:50.695149 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:50.695540 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.696027 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.696853 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:50.697071 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:50.697197 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:50.699346 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:50.699349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:50.699351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:50.699374 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:50.700619 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.701054 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.701057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.701058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.701127 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:50.701432 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:50.701435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:50.701436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:50.701622 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:50.701727 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:50.701983 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 04:34:50.702081 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 04:34:50.703411 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 20855
I20260806 04:34:50.717386 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 21055
I20260806 04:34:50.737841 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 21265
I20260806 04:34:50.757925 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 20716
2026-08-06T04:34:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2347 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[21492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[21492](info): set up 2 sockets
Aug 06 04:34:50 dist-test-slave-xt4k krb5kdc[21492](info): commencing operation
krb5kdc: starting...
W20260806 04:34:51.906831 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:51 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab.
Entry for principal oryx/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab.
I20260806 04:34:52.033676 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34829
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab
--principal=oryx/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:34829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:52.147187 21510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:52.147451 21510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:52.147507 21510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:52.151494 21510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:52.151561 21510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:52.151587 21510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:52.151610 21510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:52.151638 21510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:52.156917 21510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:34829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab
--principal=oryx/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:52.157953 21510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:52.158951 21510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:52.167305 21516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:52.167433 21521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:52.167543 21510 server_base.cc:1034] running on GCE node
W20260806 04:34:52.167305 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:52.168095 21510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:52.169160 21510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:52.170315 21510 hybrid_clock.cc:648] HybridClock initialized: now 1785990892170295 us; error 33 us; skew 500 ppm
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:52.175429 21510 init.cc:377] Logged in from keytab as oryx/127.12.15.254@KRBTEST.COM (short username oryx)
I20260806 04:34:52.176378 21510 webserver.cc:460] Webserver started at http://127.12.15.254:34271/ using document root <none> and password file <none>
I20260806 04:34:52.176921 21510 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:52.176987 21510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:52.177212 21510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:52.179281 21510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "1e456c908df943eca92a1ee10ae5a457"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "fa84501c56f8ebb3ec0148b7f8a05072"
server_key_iv: "7f77a5cfd1a3160591e404956479eff2"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.179883 21510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "1e456c908df943eca92a1ee10ae5a457"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "fa84501c56f8ebb3ec0148b7f8a05072"
server_key_iv: "7f77a5cfd1a3160591e404956479eff2"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.183736 21510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:34:52.186210 21532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.187512 21510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 04:34:52.187638 21510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "1e456c908df943eca92a1ee10ae5a457"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "fa84501c56f8ebb3ec0148b7f8a05072"
server_key_iv: "7f77a5cfd1a3160591e404956479eff2"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.187741 21510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:52.211067 21510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:52.214386 21510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:52.214586 21510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:52.222100 21510 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:34829
I20260806 04:34:52.222427 21613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:34829 every 8 connection(s)
I20260806 04:34:52.223317 21510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:52.226363 21614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:52.231284 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 21510
I20260806 04:34:52.231433 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:52.231745 12351 external_mini_cluster.cc:1428] Setting key d0ae7a367cd2c199c62b629dd28a7a58
I20260806 04:34:52.232268 21614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Bootstrap starting.
I20260806 04:34:52.234696 21614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:52.235600 21614 log.cc:824] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:52.237789 21614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: No bootstrap required, opened a new log
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:52.242067 21614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } }
I20260806 04:34:52.242314 21614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:52.242372 21614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e456c908df943eca92a1ee10ae5a457, State: Initialized, Role: FOLLOWER
I20260806 04:34:52.242835 21614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } }
I20260806 04:34:52.242970 21614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:52.243036 21614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:52.243134 21614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:52.244163 21614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } }
I20260806 04:34:52.244549 21614 leader_election.cc:302] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e456c908df943eca92a1ee10ae5a457; no voters:
I20260806 04:34:52.244891 21614 leader_election.cc:288] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:52.244995 21622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:52.245206 21622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 LEADER]: Becoming Leader. State: Replica: 1e456c908df943eca92a1ee10ae5a457, State: Running, Role: LEADER
I20260806 04:34:52.245592 21622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } }
I20260806 04:34:52.246223 21614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:52.246762 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:52.233210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.12.15.254:34829)
0806 04:34:52.233657 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:52.233672 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:52.233708 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:34:52.234345 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:52.234363 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:52.234717 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:52.235023 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:52.235874 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.237084 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.238196 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.238693 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.238816 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:52.241610 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:52.241616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:52.241631 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:52.241657 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:52.243378 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:52.243811 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:52.243816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:52.243820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:52.243931 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:52.244215 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:52.244217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:52.244218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:52.244713 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:52.244870 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:52.245515 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 04:34:52.245803 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":136,"threads_started":1}
I20260806 04:34:52.247133 21625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e456c908df943eca92a1ee10ae5a457. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } }
I20260806 04:34:52.247272 21625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:52.247634 21634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:52.247628 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } }
I20260806 04:34:52.247753 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:52.250686 21634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:52.256124 21634 catalog_manager.cc:1347] Generated new cluster ID: 96112cd7fa4446f6880d6d83cfb22b7a
I20260806 04:34:52.256191 21634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:52.283242 21634 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:52.284729 21634 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:52.302851 21634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Generated new TSK 0
I20260806 04:34:52.303625 21634 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab.
Entry for principal oryx/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab.
I20260806 04:34:52.335041 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34829
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:52.446955 21660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:52.447207 21660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:52.447264 21660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:52.451337 21660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:52.451391 21660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:52.451467 21660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:52.456561 21660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:52.457557 21660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:52.458557 21660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:52.467747 21672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:52.467742 21670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:52.467742 21669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:52.467988 21660 server_base.cc:1034] running on GCE node
I20260806 04:34:52.468498 21660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:52.469092 21660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:52.470256 21660 hybrid_clock.cc:648] HybridClock initialized: now 1785990892470243 us; error 30 us; skew 500 ppm
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:52.472909 21660 init.cc:377] Logged in from keytab as oryx/127.12.15.193@KRBTEST.COM (short username oryx)
I20260806 04:34:52.473744 21660 webserver.cc:460] Webserver started at http://127.12.15.193:39323/ using document root <none> and password file <none>
I20260806 04:34:52.474267 21660 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:52.474330 21660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:52.474536 21660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:52.476519 21660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "abaa6ef902644442aef07e3c40f04693"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "cf8b26690cf733623c57d7d12217dc52"
server_key_iv: "3bba4af05723651933187ff7b5d2daee"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.477061 21660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "abaa6ef902644442aef07e3c40f04693"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "cf8b26690cf733623c57d7d12217dc52"
server_key_iv: "3bba4af05723651933187ff7b5d2daee"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.480609 21660 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:34:52.483021 21682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.484211 21660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:52.484319 21660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "abaa6ef902644442aef07e3c40f04693"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "cf8b26690cf733623c57d7d12217dc52"
server_key_iv: "3bba4af05723651933187ff7b5d2daee"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.484463 21660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:52.510356 21660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:52.513224 21660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:52.513396 21660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:52.513974 21660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:52.514784 21660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:52.514832 21660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.514886 21660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:52.514912 21660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.523386 21660 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:36159
I20260806 04:34:52.523415 21843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:36159 every 8 connection(s)
I20260806 04:34:52.524233 21660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:52.531817 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 21660
I20260806 04:34:52.531935 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:52.532209 12351 external_mini_cluster.cc:1428] Setting key e5a10c4326dd1948167dfdfb083df678
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:52.537405 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:52.526069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:41249 (local address 127.12.15.254:34829)
0806 04:34:52.526224 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:52.526227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:52.527018 (+ 791us) server_negotiation.cc:407] Connection header received
0806 04:34:52.528388 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:52.528392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:52.528452 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:52.528528 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:52.529656 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.530187 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.531279 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.531509 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.531618 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:52.534196 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:52.534199 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:52.534201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:52.534224 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:52.535477 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:52.535975 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:52.535978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:52.535980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:52.536069 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:52.536368 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:52.536371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:52.536373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:52.536591 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:52.536695 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:52.537132 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 04:34:52.537251 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:34:52.538110 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:52.526415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:41249)
0806 04:34:52.526870 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:52.526912 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:52.528165 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:52.528704 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:52.528711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:52.529045 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:52.529510 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:52.529524 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.530304 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.530307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:52.531158 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:52.531168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.531654 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.531658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:52.531827 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:52.532365 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:52.532387 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:52.534010 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:52.535607 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:52.535613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:52.535628 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:52.535878 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:52.536146 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:52.536148 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:52.536149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:52.536267 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:52.536683 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:52.536687 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:52.536908 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 04:34:52.537060 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 04:34:52.537297 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260806 04:34:52.539302 21844 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34829
I20260806 04:34:52.539525 21844 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:52.539996 21844 heartbeater.cc:511] Master 127.12.15.254:34829 requested a full tablet report, sending...
I20260806 04:34:52.541503 21564 ts_manager.cc:194] Registered new tserver with Master: abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159)
WARNING: no policy specified for oryx/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:34:52.542659 21564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:41249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab.
Entry for principal oryx/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab.
I20260806 04:34:52.559790 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34829
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:52.671638 21855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:52.671957 21855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:52.672022 21855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:52.676117 21855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:52.676177 21855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:52.676254 21855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:52.681458 21855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:52.682516 21855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:52.683530 21855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:52.693131 21861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:52.693230 21855 server_base.cc:1034] running on GCE node
W20260806 04:34:52.693135 21862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:52.693177 21864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:52.693787 21855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:52.694403 21855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:52.695580 21855 hybrid_clock.cc:648] HybridClock initialized: now 1785990892695558 us; error 29 us; skew 500 ppm
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:52.698201 21855 init.cc:377] Logged in from keytab as oryx/127.12.15.194@KRBTEST.COM (short username oryx)
I20260806 04:34:52.699033 21855 webserver.cc:460] Webserver started at http://127.12.15.194:38363/ using document root <none> and password file <none>
I20260806 04:34:52.699642 21855 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:52.699728 21855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:52.699973 21855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:52.702091 21855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "76cf6e444f59424880d1988ca591c7d5"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "893ae4624968bfedda518fbb363de2a1"
server_key_iv: "bb99d85b6c3693e2e80e76bb4a8c07b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.702657 21855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "76cf6e444f59424880d1988ca591c7d5"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "893ae4624968bfedda518fbb363de2a1"
server_key_iv: "bb99d85b6c3693e2e80e76bb4a8c07b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.706391 21855 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:34:52.708796 21875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.710031 21855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:52.710139 21855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "76cf6e444f59424880d1988ca591c7d5"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "893ae4624968bfedda518fbb363de2a1"
server_key_iv: "bb99d85b6c3693e2e80e76bb4a8c07b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.710230 21855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:52.746098 21855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:52.749519 21855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:52.749743 21855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:52.750361 21855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:52.751236 21855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:52.751284 21855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.751338 21855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:52.751363 21855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.759902 21855 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:32835
I20260806 04:34:52.759939 22054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:32835 every 8 connection(s)
I20260806 04:34:52.760854 21855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:52.766788 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 21855
I20260806 04:34:52.766908 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:52.767233 12351 external_mini_cluster.cc:1428] Setting key a310ce48634295c7f07ba5911c17c88b
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:52.775866 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:52.762673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:43927 (local address 127.12.15.254:34829)
0806 04:34:52.762800 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:52.762803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:52.763640 (+ 837us) server_negotiation.cc:407] Connection header received
0806 04:34:52.765041 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:52.765045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:52.765106 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:52.765184 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:52.766367 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.766924 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.768391 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.768717 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.768835 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:52.771790 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:52.771794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:52.771796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:52.771821 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:52.773408 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:52.774133 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:52.774137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:52.774139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:52.774223 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:52.774599 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:52.774601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:52.774602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:52.774807 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:52.774885 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:52.775595 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 04:34:52.775726 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:34:52.777091 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:52.763035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:43927)
0806 04:34:52.763488 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:52.763530 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:52.764825 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:52.765363 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:52.765370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:52.765732 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:52.766222 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:52.766237 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.767057 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.767062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:52.768243 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:52.768254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:52.768874 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:52.768879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:52.769079 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:52.769722 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:52.769745 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:52.771577 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:52.773568 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:52.773575 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:52.773593 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:52.774008 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:52.774315 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:52.774318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:52.774320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:52.774512 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:52.774928 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:52.774936 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:52.775286 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 04:34:52.775526 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 04:34:52.775918 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for oryx/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:52.778393 22056 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34829
I20260806 04:34:52.778676 22056 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:52.779198 22056 heartbeater.cc:511] Master 127.12.15.254:34829 requested a full tablet report, sending...
I20260806 04:34:52.780589 21564 ts_manager.cc:194] Registered new tserver with Master: 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.195@KRBTEST.COM" created.
I20260806 04:34:52.781337 21564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:43927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab.
Entry for principal oryx/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab.
I20260806 04:34:52.799206 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34829
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:52.914889 22064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:52.915154 22064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:52.915196 22064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:52.919252 22064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:52.919319 22064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:52.919378 22064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:52.924705 22064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:52.925711 22064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:52.926796 22064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:52.936383 22070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:52.936400 22074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:52.936383 22069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:52.936553 22064 server_base.cc:1034] running on GCE node
I20260806 04:34:52.937165 22064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:52.937808 22064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:52.938984 22064 hybrid_clock.cc:648] HybridClock initialized: now 1785990892938964 us; error 32 us; skew 500 ppm
Aug 06 04:34:52 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:52.941846 22064 init.cc:377] Logged in from keytab as oryx/127.12.15.195@KRBTEST.COM (short username oryx)
I20260806 04:34:52.942795 22064 webserver.cc:460] Webserver started at http://127.12.15.195:43531/ using document root <none> and password file <none>
I20260806 04:34:52.943339 22064 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:52.943418 22064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:52.943629 22064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:52.945839 22064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "5859c5a9b5a64f5cb616be01df292198"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "c11327f7699ed0b852a691dc739bb132"
server_key_iv: "cb1360e3e25230e57bd8cd84d6d2f69f"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.946393 22064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "5859c5a9b5a64f5cb616be01df292198"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "c11327f7699ed0b852a691dc739bb132"
server_key_iv: "cb1360e3e25230e57bd8cd84d6d2f69f"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.950466 22064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:34:52.952943 22086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.954219 22064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:52.954335 22064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5859c5a9b5a64f5cb616be01df292198"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "c11327f7699ed0b852a691dc739bb132"
server_key_iv: "cb1360e3e25230e57bd8cd84d6d2f69f"
server_key_version: "encryptionkey@0"
I20260806 04:34:52.954439 22064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:52.984692 22064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:52.988963 22064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:52.989181 22064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:52.989827 22064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:52.990840 22064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:52.990896 22064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.990940 22064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:52.990954 22064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:52.999925 22064 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:46261
I20260806 04:34:52.999949 22264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:46261 every 8 connection(s)
I20260806 04:34:53.000921 22064 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:53.007750 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 22064
I20260806 04:34:53.007884 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:53.008174 12351 external_mini_cluster.cc:1428] Setting key eb390ddd43b4fa92788cbbf659b19b18
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:53.015182 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.002851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:38863 (local address 127.12.15.254:34829)
0806 04:34:53.003051 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.003056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.003965 (+ 909us) server_negotiation.cc:407] Connection header received
0806 04:34:53.005386 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.005390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.005443 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.005528 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.006748 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.007292 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.008614 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.008890 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.008988 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.011604 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.011609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.011611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.011642 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.013024 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.013581 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.013583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.013584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.013658 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.014057 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.014059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.014060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.014258 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.014336 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.014926 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.015059 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:34:53.016084 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.003276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:38863)
0806 04:34:53.003801 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.003845 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.005164 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.005690 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.005696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.006087 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.006590 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.006603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.007431 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.007434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.008487 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.008498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.009050 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.009055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.009267 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.009875 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.009893 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.011423 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.013177 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.013183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.013198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.013494 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.013785 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.013788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.013790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.013969 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.014348 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.014354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.014653 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.014891 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.015213 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":126,"threads_started":1}
I20260806 04:34:53.017225 22266 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34829
I20260806 04:34:53.017468 22266 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:53.017997 22266 heartbeater.cc:511] Master 127.12.15.254:34829 requested a full tablet report, sending...
I20260806 04:34:53.019243 21564 ts_manager.cc:194] Registered new tserver with Master: 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261)
I20260806 04:34:53.019827 21564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:38863
I20260806 04:34:53.022446 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.15.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.253.keytab.
Entry for principal oryx/127.12.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.253.keytab.
I20260806 04:34:53.056206 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:42907
--webserver_interface=127.12.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.253.keytab
--principal=oryx/127.12.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:34829,127.12.15.253:42907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:53.171037 22274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:53.171320 22274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:53.171378 22274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:53.175431 22274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:53.175493 22274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:53.175526 22274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:53.175549 22274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:53.175575 22274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:53.180990 22274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:34829,127.12.15.253:42907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.253.keytab
--principal=oryx/127.12.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:42907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
W20260806 04:34:53.181205 22274 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.15.254:34829,127.12.15.253:42907'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 04:34:53.182072 22274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:53.183131 22274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:53.191557 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.191565 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.191787 22274 server_base.cc:1034] running on GCE node
W20260806 04:34:53.191566 22280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.192504 22274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:53.193550 22274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:53.194721 22274 hybrid_clock.cc:648] HybridClock initialized: now 1785990893194708 us; error 39 us; skew 500 ppm
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:53.197387 22274 init.cc:377] Logged in from keytab as oryx/127.12.15.253@KRBTEST.COM (short username oryx)
I20260806 04:34:53.198276 22274 webserver.cc:460] Webserver started at http://127.12.15.253:42911/ using document root <none> and password file <none>
I20260806 04:34:53.198789 22274 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.198845 22274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.199004 22274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:53.201002 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/instance:
uuid: "329a4b2657dd4e629ea186a2946a12c6"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "94d56603bb1feb2356abbba1f38791c3"
server_key_iv: "847da30e357581bdbb857c1a5ad42a78"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.201489 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal/instance:
uuid: "329a4b2657dd4e629ea186a2946a12c6"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "94d56603bb1feb2356abbba1f38791c3"
server_key_iv: "847da30e357581bdbb857c1a5ad42a78"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.205255 22274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:34:53.207540 22289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.208801 22274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:53.208901 22274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
uuid: "329a4b2657dd4e629ea186a2946a12c6"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "94d56603bb1feb2356abbba1f38791c3"
server_key_iv: "847da30e357581bdbb857c1a5ad42a78"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.208984 22274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.221715 22274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:53.225060 22274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:53.225255 22274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:53.232910 22274 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.253:42907
I20260806 04:34:53.233258 22372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.253:42907 every 8 connection(s)
I20260806 04:34:53.233945 22274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
I20260806 04:34:53.236819 22377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:53.240622 22377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:34:53.243826 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 22274
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.253@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
I20260806 04:34:53.255189 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.242573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.12.15.254:34829)
0806 04:34:53.242751 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.242757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.243548 (+ 791us) server_negotiation.cc:407] Connection header received
0806 04:34:53.244868 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.244871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.244932 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.245010 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.246197 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.246813 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.247975 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.248319 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.248454 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.251091 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.251095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.251097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.251124 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.252840 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.253367 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.253371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.253372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.253467 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.253907 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.253911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.253913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.254224 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.254301 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.254881 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.255020 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:34:53.256096 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.244867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.12.15.253:42907)
0806 04:34:53.245289 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.245301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.245323 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:34:53.245358 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.245361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.245513 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.245652 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.246483 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.247461 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.248363 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.248726 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.248824 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.251512 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.251520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.251537 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.251564 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.253428 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.253945 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.253952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.253953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.254036 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.254347 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.254349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.254351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.254628 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.254794 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.255044 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.255418 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":324,"spinlock_wait_cycles":4352,"thread_start_us":175,"threads_started":1}
I20260806 04:34:53.256098 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.242917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:58352)
0806 04:34:53.243391 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.243432 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.244571 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.245224 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.245231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.245583 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.246056 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.246069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.246937 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.246939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.247864 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.247874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.248493 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.248497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.248691 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.249284 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.249305 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.250859 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.252963 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.252971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.252981 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.253270 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.253580 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.253584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.253586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.253795 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.254339 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.254346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.254626 (+ 280us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.254821 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.255202 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":43,"thread_start_us":122,"threads_started":1}
I20260806 04:34:53.257542 22377 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.253@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
I20260806 04:34:53.265606 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.258074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:56548)
0806 04:34:53.258190 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.258204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.258273 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.258502 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.258505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.258676 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.258770 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.258776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.259524 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.259526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.260043 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.260051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.261032 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.261035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.261113 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.261549 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.261561 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.262825 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.264446 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.264449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.264450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.264652 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.264884 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.264886 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.264887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.264957 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.265237 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.265239 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.265312 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.265384 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.265481 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 04:34:53.265758 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.258218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.12.15.253:42907)
0806 04:34:53.258320 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.258323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.258334 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:34:53.258361 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.258362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.258403 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.258464 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.258872 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.259385 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.260148 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.260913 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.260999 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.262970 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.262972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.262975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.262996 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.264309 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.264739 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.264742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.264743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.264816 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.265019 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.265021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.265022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.265175 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.265243 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.265504 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.265620 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 04:34:53.268553 22377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Bootstrap starting.
I20260806 04:34:53.270792 22377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:53.271493 22377 log.cc:824] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:53.273443 22377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: No bootstrap required, opened a new log
I20260806 04:34:53.276747 22377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } }
I20260806 04:34:53.276986 22377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:53.277034 22377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329a4b2657dd4e629ea186a2946a12c6, State: Initialized, Role: FOLLOWER
I20260806 04:34:53.277448 22377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } }
I20260806 04:34:53.277949 22387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } } }
I20260806 04:34:53.278079 22387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:53.278857 22377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 04:34:53.283824 22405 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:34:53.283914 22405 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:34:53.285655 22274 master_runner.cc:428] Detected that this master 329a4b2657dd4e629ea186a2946a12c6 is joining an existing cluster
I20260806 04:34:53.285709 22274 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.15.253:42907
I20260806 04:34:53.297122 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.288305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.12.15.253:42907)
0806 04:34:53.288415 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.288419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.288664 (+ 245us) server_negotiation.cc:407] Connection header received
0806 04:34:53.288790 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.288793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.288831 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.288890 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.289401 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.290367 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.291644 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.291848 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.291897 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.293859 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.293863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.293866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.293886 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.295536 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.295961 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.295964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.295966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.296058 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.296307 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.296310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.296312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.296517 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.296603 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.296833 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.296978 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":48}
I20260806 04:34:53.297137 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.288220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:56562)
0806 04:34:53.288581 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.288594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.288712 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.288913 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.288916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.289177 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.289296 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.289303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.290470 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.290473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.291523 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.291534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.292027 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.292029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.292101 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.293099 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.293113 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.293731 (+ 618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.295635 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.295638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.295640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.295846 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.296144 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.296147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.296149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.296235 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.296602 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.296605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.296689 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.296780 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.296925 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":95,"spinlock_wait_cycles":6784,"thread_start_us":166,"threads_started":1}
I20260806 04:34:53.297747 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.287861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.12.15.254:34829)
0806 04:34:53.287982 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.287986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.288235 (+ 249us) server_negotiation.cc:407] Connection header received
0806 04:34:53.288646 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.288650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.288705 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.288790 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.289451 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.290343 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.291665 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.292003 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.292124 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.293879 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.293883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.293887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.293914 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.295823 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.296197 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.296201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.296203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.296308 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.296638 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.296642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.296643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.296847 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.296914 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.297469 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.297594 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:34:53.298447 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.287750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:58368)
0806 04:34:53.288135 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.288150 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.288313 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.288919 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.288922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.289216 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.289344 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.289352 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.290470 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.290473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.291528 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.291534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.292163 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.292166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.292245 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.293061 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.293077 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.293731 (+ 654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.295924 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.295927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.295929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.296118 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.296445 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.296448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.296450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.296538 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.296985 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.296988 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.297093 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.298115 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.298256 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":29,"spinlock_wait_cycles":4352,"thread_start_us":110,"threads_started":1}
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.254@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
I20260806 04:34:53.319051 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.303010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.12.15.253:42907)
0806 04:34:53.303120 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.303124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.308874 (+ 5750us) server_negotiation.cc:407] Connection header received
0806 04:34:53.309149 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.309152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.309216 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.309296 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.310210 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.310886 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.312128 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.312482 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.312588 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.315327 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.315332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.315335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.315363 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.317044 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.317618 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.317622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.317624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.317713 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.318017 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.318020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.318022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.318241 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.318336 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.318760 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.318880 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:34:53.319128 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.302929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:56574)
0806 04:34:53.308687 (+ 5758us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.308771 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.308949 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.309362 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.309370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.309899 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.310062 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.310072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.310996 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.310999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.312007 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.312019 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.312596 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.312599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.312723 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.313239 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.313274 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.315179 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.317164 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.317175 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.317183 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.317513 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.317809 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.317812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.317814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.317928 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.318352 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.318360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.318529 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.318704 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.318943 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5560,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.12.15.252@KRBTEST.COM; defaulting to no policy
I20260806 04:34:53.321131 21564 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.15.253:42907: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6"
member_type: NON_VOTER
last_known_addr {
host: "127.12.15.253"
port: 42907
}
attrs {
promote: true
}
}
dest_uuid: "1e456c908df943eca92a1ee10ae5a457"
cas_config_opid_index: -1
I20260806 04:34:53.321859 21564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.252@KRBTEST.COM" created.
I20260806 04:34:53.323828 22419 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 329a4b2657dd4e629ea186a2946a12c6 (127.12.15.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } }
I20260806 04:34:53.324550 22325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:53.325762 22419 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.15.253:42907
I20260806 04:34:53.325973 22325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Refusing update from remote peer 1e456c908df943eca92a1ee10ae5a457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 04:34:53.326534 22274 master.cc:537] Master@127.12.15.253:42907 shutting down...
I20260806 04:34:53.326535 22420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } }
I20260806 04:34:53.326637 22420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:53.328198 22420 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:34:53.330797 21544 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 04:34:53.332857 22274 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:53.333158 22274 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:53.333262 22274 tablet_replica.cc:330] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: stopping tablet replica
W20260806 04:34:53.334227 21544 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 -> Peer 329a4b2657dd4e629ea186a2946a12c6 (127.12.15.253:42907): Couldn't send request to peer 329a4b2657dd4e629ea186a2946a12c6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.252.keytab.
Entry for principal oryx/127.12.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.252.keytab.
I20260806 04:34:53.344462 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:38331
--webserver_interface=127.12.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.252.keytab
--principal=oryx/127.12.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:34:53.349807 22274 master.cc:552] Master@127.12.15.253:42907 shutdown complete.
I20260806 04:34:53.349874 22274 master_runner.cc:305] Clearing existing system tablet
I20260806 04:34:53.350124 22274 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.350179 22274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.351539 22274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 04:34:53.352502 22427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.352748 22274 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.352877 22274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:53.352916 22274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
uuid: "329a4b2657dd4e629ea186a2946a12c6"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "94d56603bb1feb2356abbba1f38791c3"
server_key_iv: "847da30e357581bdbb857c1a5ad42a78"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.355122 22274 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:53.357556 22274 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 04:34:53.357648 22274 log.cc:1196] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 04:34:53.357890 22274 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Deleting consensus metadata
I20260806 04:34:53.358057 22274 master_runner.cc:315] Copying system tablet from 127.12.15.254:34829
I20260806 04:34:53.359290 22274 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: tablet copy: Beginning tablet copy session from remote peer at address 127.12.15.254:34829
I20260806 04:34:53.368489 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.359994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.12.15.254:34829)
0806 04:34:53.360120 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.360125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.360308 (+ 183us) server_negotiation.cc:407] Connection header received
0806 04:34:53.360498 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.360501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.360553 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.360642 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.361170 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.361975 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.363221 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.363567 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.363665 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.364961 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.364965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.364968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.364996 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.366703 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.367187 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.367190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.367192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.367280 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.367570 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.367573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.367575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.367790 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.367884 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.368152 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.368273 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 04:34:53.368783 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.359899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:58370)
0806 04:34:53.360212 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.360227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.360388 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.360679 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.360682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.360894 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.361039 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.361049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.362084 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.362088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.363103 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.363114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.363683 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.363687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.363789 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.364362 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.364383 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.364815 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.366828 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.366832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.366834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.367092 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.367365 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.367367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.367369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.367477 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.367881 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.367884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.367989 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.368093 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.368240 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":86,"threads_started":1}
I20260806 04:34:53.369349 21595 tablet_copy_service.cc:140] P 1e456c908df943eca92a1ee10ae5a457: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 329a4b2657dd4e629ea186a2946a12c6 ({username='oryx', principal='oryx/127.12.15.253@KRBTEST.COM'} at 127.0.0.1:58370)
I20260806 04:34:53.369534 21595 tablet_copy_service.cc:161] P 1e456c908df943eca92a1ee10ae5a457: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 329a4b2657dd4e629ea186a2946a12c6 at {username='oryx', principal='oryx/127.12.15.253@KRBTEST.COM'} at 127.0.0.1:58370: session id = 329a4b2657dd4e629ea186a2946a12c6-00000000000000000000000000000000
I20260806 04:34:53.371251 21595 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 04:34:53.373049 22274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 04:34:53.375402 22274 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: tablet copy: Starting download of 0 data blocks...
I20260806 04:34:53.375586 22274 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: tablet copy: Starting download of 1 WAL segments...
I20260806 04:34:53.377382 22274 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 04:34:53.379179 21595 tablet_copy_service.cc:342] P 1e456c908df943eca92a1ee10ae5a457: Request end of tablet copy session 329a4b2657dd4e629ea186a2946a12c6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.15.253@KRBTEST.COM'} at 127.0.0.1:58370
I20260806 04:34:53.379356 21595 tablet_copy_service.cc:433] P 1e456c908df943eca92a1ee10ae5a457: ending tablet copy session 329a4b2657dd4e629ea186a2946a12c6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 329a4b2657dd4e629ea186a2946a12c6
I20260806 04:34:53.387817 22274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:53.389734 22435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.393321 22436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.396489 22274 server_base.cc:1034] running on GCE node
W20260806 04:34:53.397284 22438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.397552 22274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:53.397881 22274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:53.399036 22274 hybrid_clock.cc:648] HybridClock initialized: now 1785990893399017 us; error 38 us; skew 500 ppm
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:53.400789 22274 init.cc:377] Logged in from keytab as oryx/127.12.15.253@KRBTEST.COM (short username oryx)
I20260806 04:34:53.401285 22274 webserver.cc:460] Webserver started at http://127.12.15.253:35967/ using document root <none> and password file <none>
I20260806 04:34:53.401494 22274 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.401539 22274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.402946 22274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:53.404016 22447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.404261 22274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:53.404384 22274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
uuid: "329a4b2657dd4e629ea186a2946a12c6"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "94d56603bb1feb2356abbba1f38791c3"
server_key_iv: "847da30e357581bdbb857c1a5ad42a78"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.404567 22274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.424455 22274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:53.426590 22274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:53.431682 22274 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.253:42907
I20260806 04:34:53.432410 22274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
I20260806 04:34:53.434341 22526 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:34:53.435626 22526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Bootstrap starting.
I20260806 04:34:53.436488 22525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.253:42907 every 8 connection(s)
I20260806 04:34:53.446218 22526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:34:53.446525 22526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: Bootstrap complete.
I20260806 04:34:53.447167 22526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } }
I20260806 04:34:53.447302 22526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 329a4b2657dd4e629ea186a2946a12c6, State: Initialized, Role: LEARNER
I20260806 04:34:53.447408 22526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } }
I20260806 04:34:53.447742 22531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } }
I20260806 04:34:53.447901 22526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:53.447921 22531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: LEARNER
I20260806 04:34:53.451061 22544 catalog_manager.cc:1259] Loaded cluster ID: 96112cd7fa4446f6880d6d83cfb22b7a
I20260806 04:34:53.451123 22544 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: loading cluster ID for follower catalog manager: success
I20260806 04:34:53.453341 22544 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: acquiring CA information for follower catalog manager: success
I20260806 04:34:53.454847 22544 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 04:34:53.478711 22425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:53.478948 22425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:53.478986 22425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:53.482863 22425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:53.482911 22425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:53.482929 22425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:53.482944 22425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:53.482957 22425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:53.488123 22425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.252.keytab
--principal=oryx/127.12.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:38331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:53.489117 22425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:53.490085 22425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:53.498420 22553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.498535 22555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.498577 22425 server_base.cc:1034] running on GCE node
W20260806 04:34:53.498425 22558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.499136 22425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:53.500191 22425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:53.501353 22425 hybrid_clock.cc:648] HybridClock initialized: now 1785990893501321 us; error 44 us; skew 500 ppm
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:53.504025 22425 init.cc:377] Logged in from keytab as oryx/127.12.15.252@KRBTEST.COM (short username oryx)
I20260806 04:34:53.504972 22425 webserver.cc:460] Webserver started at http://127.12.15.252:35001/ using document root <none> and password file <none>
I20260806 04:34:53.505508 22425 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.505594 22425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.505826 22425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:53.507870 22425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/instance:
uuid: "ccc2dc9bff23409b807eb0ea5765e6dc"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "cfd874062641c17a9ea1a9bd097ee121"
server_key_iv: "21120dd663f9174cea2c1144063388a4"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.508445 22425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal/instance:
uuid: "ccc2dc9bff23409b807eb0ea5765e6dc"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "cfd874062641c17a9ea1a9bd097ee121"
server_key_iv: "21120dd663f9174cea2c1144063388a4"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.512354 22425 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:34:53.514882 22568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.516081 22425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:53.516196 22425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
uuid: "ccc2dc9bff23409b807eb0ea5765e6dc"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "cfd874062641c17a9ea1a9bd097ee121"
server_key_iv: "21120dd663f9174cea2c1144063388a4"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.516290 22425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.534480 22425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:53.537560 22425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:53.537731 22425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:53.544674 22425 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.252:38331
I20260806 04:34:53.544938 21844 heartbeater.cc:503] Master 127.12.15.254:34829 was elected leader, sending a full tablet report...
I20260806 04:34:53.544963 22651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.252:38331 every 8 connection(s)
I20260806 04:34:53.545769 22425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
I20260806 04:34:53.548656 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:53.552161 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 22425
I20260806 04:34:53.552678 22652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.252@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.252@KRBTEST.COM
I20260806 04:34:53.565007 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.554504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.12.15.254:34829)
0806 04:34:53.554628 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.554632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.554848 (+ 216us) server_negotiation.cc:407] Connection header received
0806 04:34:53.555165 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.555168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.555219 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.555298 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.556305 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.557144 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.558358 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.558586 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.558688 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.561223 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.561227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.561230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.561254 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.562652 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.563226 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.563229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.563230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.563303 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.563763 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.563767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.563769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.564039 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.564155 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.564760 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.564858 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:34:53.566274 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.554334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:58380)
0806 04:34:53.554713 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.554760 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.554993 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.555339 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.555346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.555795 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.556169 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.556188 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.557261 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.557264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.558248 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.558259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.558730 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.558733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.558902 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.559453 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.559475 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.561090 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.562758 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.562768 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.562778 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.563137 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.563387 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.563390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.563392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.563556 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.564186 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.564192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.564524 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.564710 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.565285 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":123,"threads_started":1}
I20260806 04:34:53.566272 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.553504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.12.15.252:38331)
0806 04:34:53.553918 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.553935 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.553972 (+ 37us) server_negotiation.cc:407] Connection header received
0806 04:34:53.554688 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.554711 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.554995 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.555463 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.556265 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.557542 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.558634 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.559088 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.559188 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.561834 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.561840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.561849 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.561871 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.563610 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.564186 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.564192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.564194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.564293 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.564622 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.564625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.564627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.564899 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.565121 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.565309 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.565603 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
I20260806 04:34:53.568216 22652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.252@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
I20260806 04:34:53.575947 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.568692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:56576)
0806 04:34:53.568818 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.568831 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.568905 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.569307 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.569310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.569444 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.569569 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.569574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.570433 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.570436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.570997 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.571005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.571458 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.571461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.571566 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.571968 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.571980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.573173 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.574802 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.574805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.574807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.575005 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.575216 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.575218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.575219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.575284 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.575569 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.575571 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.575645 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.575725 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.575823 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 04:34:53.576122 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.568826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.12.15.253:42907)
0806 04:34:53.569091 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.569094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.569106 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:34:53.569142 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.569144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.569182 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.569253 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.569669 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.570339 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.571106 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.571345 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.571434 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.573314 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.573317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.573319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.573343 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.574677 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.575083 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.575085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.575086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.575155 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.575342 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.575344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.575345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.575503 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.575572 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.575837 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.575947 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":111,"threads_started":1}
I20260806 04:34:53.576821 22652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.252@KRBTEST.COM for oryx/127.12.15.252@KRBTEST.COM
I20260806 04:34:53.583510 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.577319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34214)
0806 04:34:53.577430 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.577442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.577502 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.577686 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.577688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.577811 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.577877 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.577882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.578565 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.578567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.579059 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.579066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.579420 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.579422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.579474 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.579859 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.579869 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.581022 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.582428 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.582430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.582432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.582612 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.582817 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.582819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.582820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.582884 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.583146 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.583147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.583215 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.583283 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.583372 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 04:34:53.583537 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.577431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.12.15.252:38331)
0806 04:34:53.577531 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.577534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.577544 (+ 10us) server_negotiation.cc:407] Connection header received
0806 04:34:53.577571 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.577572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.577604 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.577662 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.577955 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.578451 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.579139 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.579347 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.579415 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.581144 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.581147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.581149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.581169 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.582321 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.582683 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.582686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.582687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.582754 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.582940 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.582942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.582943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.583086 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.583152 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.583332 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.583414 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 04:34:53.586850 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Bootstrap starting.
I20260806 04:34:53.589247 22652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:53.590157 22652 log.cc:824] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:53.592092 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: No bootstrap required, opened a new log
I20260806 04:34:53.595203 22652 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } }
I20260806 04:34:53.595403 22652 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:53.595453 22652 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccc2dc9bff23409b807eb0ea5765e6dc, State: Initialized, Role: FOLLOWER
I20260806 04:34:53.595867 22652 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } }
I20260806 04:34:53.596388 22658 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } } }
I20260806 04:34:53.596515 22658 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:53.597146 22652 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: configured and running, proceeding with master startup.
W20260806 04:34:53.601868 22676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:34:53.601954 22676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:34:53.605851 22425 master_runner.cc:428] Detected that this master ccc2dc9bff23409b807eb0ea5765e6dc is joining an existing cluster
I20260806 04:34:53.605921 22425 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.15.252:38331
I20260806 04:34:53.617355 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.609094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34230 (local address 127.12.15.252:38331)
0806 04:34:53.609601 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.609605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.609629 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:34:53.609663 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.609665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.609702 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.609762 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.610571 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.611329 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.612904 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.613190 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.613245 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.614601 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.614605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.614607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.614631 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.615898 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.616263 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.616266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.616268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.616364 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.616600 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.616603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.616605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.616770 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.616850 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.617075 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.617214 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":450}
I20260806 04:34:53.617368 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.608604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34230)
0806 04:34:53.609476 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.609489 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.609573 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.610031 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.610034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.610280 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.610464 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.610469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.611445 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.611448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.612813 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.612822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.613379 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.613382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.613544 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.614090 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.614103 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.614471 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.615995 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.615998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.616000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.616184 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.616434 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.616437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.616439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.616524 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.616850 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.616853 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.616936 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.617021 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.617164 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":152,"spinlock_wait_cycles":13056,"thread_start_us":79,"threads_started":1}
I20260806 04:34:53.620321 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 21660
I20260806 04:34:53.655233 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.609076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.12.15.253:42907)
0806 04:34:53.609199 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.609203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.609641 (+ 438us) server_negotiation.cc:407] Connection header received
0806 04:34:53.609794 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.609798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.609851 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.609932 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.610627 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.611420 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.612941 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.613332 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.613406 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.650818 (+ 37412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.650823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.650826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.650858 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.653126 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.653720 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.653723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.653725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.653831 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.654163 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.654167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.654169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.654432 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.654504 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.654919 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.655058 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 04:34:53.655233 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.608990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.12.15.254:34829)
0806 04:34:53.609213 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.609217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.609232 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:34:53.609281 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.609284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.609333 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.609416 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.610593 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.611359 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.612800 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.613140 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.613210 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.650818 (+ 37608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.650823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.650826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.650858 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.652561 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.653719 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.653723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.653725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.653831 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.654164 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.654167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.654169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.654421 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.654499 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.654919 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.655057 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260806 04:34:53.655783 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.608549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:58388)
0806 04:34:53.608866 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.608882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.608975 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.609848 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.609850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.610261 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.610457 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.610465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.611474 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.611477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.612667 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.612678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.613376 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.613380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.613521 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.649809 (+ 36288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.649834 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.650577 (+ 743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.653250 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.653253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.653255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.653595 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.653947 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.653950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.653952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.654058 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.654566 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.654568 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.654713 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.655411 (+ 698us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.655585 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":107,"spinlock_wait_cycles":25856,"thread_start_us":109,"threads_started":1}
I20260806 04:34:53.655817 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.609196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:56580)
0806 04:34:53.609562 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.609576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.609692 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.609955 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.609958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.610268 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.610484 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.610493 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.611561 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.611564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.612805 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.612816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.613624 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.613628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.613763 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.649918 (+ 36155us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.649930 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.650577 (+ 647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.653250 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.653253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.653255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.653595 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.653941 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.653944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.653946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.654052 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.654557 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.654560 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.654713 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.655549 (+ 836us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.655663 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":91,"spinlock_wait_cycles":7936,"thread_start_us":95,"threads_started":1}
I20260806 04:34:53.659770 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:36159
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:34829
--webserver_port=39323
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990892, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.254@KRBTEST.COM for oryx/127.12.15.252@KRBTEST.COM
I20260806 04:34:53.669883 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.661831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34234)
0806 04:34:53.662081 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.662097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.662176 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.662813 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.662815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.662985 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.663082 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.663088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.663776 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.663780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.664310 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.664317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.664838 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.664842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.664933 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.665341 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.665354 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.666643 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.668435 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.668438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.668440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.668637 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.668897 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.668900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.668902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.668988 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.669363 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.669366 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.669464 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.669551 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.669711 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260806 04:34:53.669883 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.662334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.12.15.252:38331)
0806 04:34:53.662465 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.662469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.662482 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:53.662562 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.662564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.662621 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.662693 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.663191 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.663679 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.664437 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.664734 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.664826 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.666775 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.666778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.666779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.666800 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.668220 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.668733 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.668736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.668737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.668812 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.669078 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.669081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.669083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.669271 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.669355 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.669614 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.669720 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 04:34:53.671141 21564 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.15.252:38331: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc"
member_type: NON_VOTER
last_known_addr {
host: "127.12.15.252"
port: 38331
}
attrs {
promote: true
}
}
dest_uuid: "1e456c908df943eca92a1ee10ae5a457"
cas_config_opid_index: 5
I20260806 04:34:53.671741 21564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } }
I20260806 04:34:53.672895 22420 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.673383 22611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:53.674456 22420 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.15.252:38331
I20260806 04:34:53.675215 22425 master.cc:537] Master@127.12.15.252:38331 shutting down...
I20260806 04:34:53.675338 22611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Refusing update from remote peer 1e456c908df943eca92a1ee10ae5a457: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 04:34:53.677591 22419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.677717 22419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:53.678532 22419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:34:53.681602 21536 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 -> Peer ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252:38331): Couldn't send request to peer ccc2dc9bff23409b807eb0ea5765e6dc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 04:34:53.682637 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.676818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:56592)
0806 04:34:53.678248 (+ 1430us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.678264 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.678353 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.678814 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.678817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.678835 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:53.678928 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.678934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.679927 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.679930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.681136 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.681148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.682221 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.682225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.682310 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.682359 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.682450 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.682530 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291}
I20260806 04:34:53.683234 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.677151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.12.15.253:42907)
0806 04:34:53.677904 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.677909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.678436 (+ 527us) server_negotiation.cc:407] Connection header received
0806 04:34:53.678508 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.678511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.678577 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.678669 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:53.679072 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.679799 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.681273 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.681881 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.681981 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.682159 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.682839 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.682924 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659}
I20260806 04:34:53.683854 22474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 LEARNER]: Refusing update from remote peer 1e456c908df943eca92a1ee10ae5a457: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 04:34:53.683923 22425 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:53.684250 22425 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:53.684406 22425 tablet_replica.cc:330] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: stopping tablet replica
I20260806 04:34:53.684588 22689 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 04:34:53.686659 22474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.689389 22692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.689582 22692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: LEARNER
I20260806 04:34:53.689850 22531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e456c908df943eca92a1ee10ae5a457. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: NON_VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: true } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.689937 22531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: LEARNER
I20260806 04:34:53.690763 22425 master.cc:552] Master@127.12.15.252:38331 shutdown complete.
I20260806 04:34:53.690814 22425 master_runner.cc:305] Clearing existing system tablet
I20260806 04:34:53.691028 22425 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.691085 22425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.692267 22425 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 04:34:53.693104 22695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.693284 22425 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.693405 22425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:53.693444 22425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
uuid: "ccc2dc9bff23409b807eb0ea5765e6dc"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "cfd874062641c17a9ea1a9bd097ee121"
server_key_iv: "21120dd663f9174cea2c1144063388a4"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.695143 22425 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:53.697551 22425 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 04:34:53.697650 22425 log.cc:1196] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 04:34:53.697887 22425 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Deleting consensus metadata
I20260806 04:34:53.698063 22425 master_runner.cc:315] Copying system tablet from 127.12.15.254:34829
I20260806 04:34:53.698498 22690 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: attempting to promote NON_VOTER 329a4b2657dd4e629ea186a2946a12c6 to VOTER
I20260806 04:34:53.699079 22690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } }
I20260806 04:34:53.699568 22425 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: tablet copy: Beginning tablet copy session from remote peer at address 127.12.15.254:34829
I20260806 04:34:53.700837 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.700555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34246)
0806 04:34:53.700670 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.700752 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:38331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 04:34:53.701105 22474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 LEARNER]: Refusing update from remote peer 1e456c908df943eca92a1ee10ae5a457: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 04:34:53.701609 22689 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 04:34:53.702001 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.701695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34256)
0806 04:34:53.701884 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.701936 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:38331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 04:34:53.702193 21536 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 -> Peer ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252:38331): Couldn't send request to peer ccc2dc9bff23409b807eb0ea5765e6dc. Status: Network error: Client connection negotiation failed: client connection to 127.12.15.252:38331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 04:34:53.703275 22690 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 329a4b2657dd4e629ea186a2946a12c6 (127.12.15.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.704238 22474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 329a4b2657dd4e629ea186a2946a12c6 (127.12.15.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.705642 22531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.705755 22531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:53.706938 22419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.707022 22689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.707108 22689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:53.707068 22419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:53.719113 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.708758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.12.15.254:34829)
0806 04:34:53.708870 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.708874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.712600 (+ 3726us) server_negotiation.cc:407] Connection header received
0806 04:34:53.712743 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.712746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.712797 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.712866 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.713336 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.713920 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.714687 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.714951 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.715043 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.716106 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.716110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.716112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.716138 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.717552 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.717946 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.717950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.717952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.718046 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.718324 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.718328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.718330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.718519 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.718604 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.718855 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.718963 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:34:53.719115 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.708698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:58398)
0806 04:34:53.712501 (+ 3803us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.712517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.712640 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.712897 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.712900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.713122 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.713215 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.713222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.714036 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.714039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.714585 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.714593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.715058 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.715062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.715153 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.715638 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.715651 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.715966 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.717654 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.717657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.717660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.717860 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.718152 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.718155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.718156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.718248 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.718606 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.718609 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.718704 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.718789 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.718935 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3722,"thread_start_us":106,"threads_started":1}
I20260806 04:34:53.719785 21595 tablet_copy_service.cc:140] P 1e456c908df943eca92a1ee10ae5a457: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ccc2dc9bff23409b807eb0ea5765e6dc ({username='oryx', principal='oryx/127.12.15.252@KRBTEST.COM'} at 127.0.0.1:58398)
I20260806 04:34:53.719877 21595 tablet_copy_service.cc:161] P 1e456c908df943eca92a1ee10ae5a457: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ccc2dc9bff23409b807eb0ea5765e6dc at {username='oryx', principal='oryx/127.12.15.252@KRBTEST.COM'} at 127.0.0.1:58398: session id = ccc2dc9bff23409b807eb0ea5765e6dc-00000000000000000000000000000000
I20260806 04:34:53.720785 21595 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 04:34:53.721873 22425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 04:34:53.724032 22425 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: tablet copy: Starting download of 0 data blocks...
I20260806 04:34:53.724330 22425 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: tablet copy: Starting download of 1 WAL segments...
I20260806 04:34:53.725701 22425 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 04:34:53.727196 21595 tablet_copy_service.cc:342] P 1e456c908df943eca92a1ee10ae5a457: Request end of tablet copy session ccc2dc9bff23409b807eb0ea5765e6dc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.15.252@KRBTEST.COM'} at 127.0.0.1:58398
I20260806 04:34:53.727295 21595 tablet_copy_service.cc:433] P 1e456c908df943eca92a1ee10ae5a457: ending tablet copy session ccc2dc9bff23409b807eb0ea5765e6dc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ccc2dc9bff23409b807eb0ea5765e6dc
I20260806 04:34:53.734136 22425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:53.735627 22706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.735846 22710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.736110 22707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.736392 22425 server_base.cc:1034] running on GCE node
I20260806 04:34:53.736572 22425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:53.736840 22425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:53.738003 22425 hybrid_clock.cc:648] HybridClock initialized: now 1785990893737986 us; error 25 us; skew 500 ppm
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:53.739524 22425 init.cc:377] Logged in from keytab as oryx/127.12.15.252@KRBTEST.COM (short username oryx)
I20260806 04:34:53.739965 22425 webserver.cc:460] Webserver started at http://127.12.15.252:36505/ using document root <none> and password file <none>
I20260806 04:34:53.740147 22425 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.740182 22425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.741294 22425 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:53.742162 22722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.742370 22425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:53.742453 22425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
uuid: "ccc2dc9bff23409b807eb0ea5765e6dc"
format_stamp: "Formatted at 2026-08-06 04:34:53 on dist-test-slave-xt4k"
server_key: "cfd874062641c17a9ea1a9bd097ee121"
server_key_iv: "21120dd663f9174cea2c1144063388a4"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.742537 22425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.753528 22425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:53.755295 22425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:53.759964 22425 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.252:38331
I20260806 04:34:53.760555 22425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
I20260806 04:34:53.761013 22800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.252:38331 every 8 connection(s)
I20260806 04:34:53.762852 22803 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:34:53.763934 22803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Bootstrap starting.
I20260806 04:34:53.772957 22803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:34:53.773232 22803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Bootstrap complete.
I20260806 04:34:53.773729 22803 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } }
I20260806 04:34:53.773846 22803 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ccc2dc9bff23409b807eb0ea5765e6dc, State: Initialized, Role: LEARNER
I20260806 04:34:53.773958 22803 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } }
I20260806 04:34:53.774274 22810 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:53.774451 22803 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:53.774453 22810 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: This master's current role is: LEARNER
I20260806 04:34:53.777081 22824 catalog_manager.cc:1259] Loaded cluster ID: 96112cd7fa4446f6880d6d83cfb22b7a
I20260806 04:34:53.777187 22824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: loading cluster ID for follower catalog manager: success
I20260806 04:34:53.779034 22824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: acquiring CA information for follower catalog manager: success
I20260806 04:34:53.780158 22824 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:34:53.783614 22056 heartbeater.cc:503] Master 127.12.15.254:34829 was elected leader, sending a full tablet report...
W20260806 04:34:53.817035 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:53.817302 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:53.817344 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:53.821296 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:53.821357 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:53.821414 22688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:53.827016 22688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:36159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=39323
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:53.828125 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:53.829169 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:53.838898 22830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.838901 22831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:53.838901 22833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:53.838946 22688 server_base.cc:1034] running on GCE node
I20260806 04:34:53.839541 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:53.840139 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:53.841316 22688 hybrid_clock.cc:648] HybridClock initialized: now 1785990893841281 us; error 43 us; skew 500 ppm
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:53.844619 22688 init.cc:377] Logged in from keytab as oryx/127.12.15.193@KRBTEST.COM (short username oryx)
I20260806 04:34:53.845546 22688 webserver.cc:460] Webserver started at http://127.12.15.193:39323/ using document root <none> and password file <none>
I20260806 04:34:53.846097 22688 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:53.846184 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:53.850337 22688 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:53.852509 22844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.853859 22688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:53.853984 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "abaa6ef902644442aef07e3c40f04693"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "cf8b26690cf733623c57d7d12217dc52"
server_key_iv: "3bba4af05723651933187ff7b5d2daee"
server_key_version: "encryptionkey@0"
I20260806 04:34:53.854621 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:53.881009 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:53.884131 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:53.884315 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:53.884984 22688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:53.885905 22688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:53.885952 22688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.885996 22688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:53.886025 22688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:53.897387 22688 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:36159
I20260806 04:34:53.897421 23021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:36159 every 8 connection(s)
I20260806 04:34:53.898396 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:53.905500 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 22688
I20260806 04:34:53.905704 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 21855
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.252@KRBTEST.COM
I20260806 04:34:53.915294 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.901847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:50765 (local address 127.12.15.253:42907)
0806 04:34:53.901962 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.901965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.902023 (+ 58us) server_negotiation.cc:407] Connection header received
0806 04:34:53.902533 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.902537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.902593 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.902644 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.903956 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.904824 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.908704 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.908969 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.909037 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.911776 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.911780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.911782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.911806 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.913243 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.913782 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.913785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.913786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.913862 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.914184 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.914187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.914188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.914376 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.914434 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.915001 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.915134 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:34:53.916803 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.901701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.12.15.193:50765)
0806 04:34:53.901941 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.901955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.902310 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.902830 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.902837 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.903315 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.903804 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.903811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.904975 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.904979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.908581 (+ 3602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.908592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.909215 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.909218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.909395 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.909996 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.910015 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.911607 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.913357 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.913362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.913377 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.913672 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.913947 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.913949 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.913951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.914103 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.914500 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.914506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.914779 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.915507 (+ 728us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.915856 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":132,"spinlock_wait_cycles":12032,"thread_start_us":79,"threads_started":1}
I20260806 04:34:53.918284 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.900372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:43937 (local address 127.12.15.252:38331)
0806 04:34:53.900646 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.900650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.901338 (+ 688us) server_negotiation.cc:407] Connection header received
0806 04:34:53.902506 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.902509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.902567 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.902645 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.903955 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.904871 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.908879 (+ 4008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.909305 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.909391 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.913405 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.913409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.913412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.913443 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.915434 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.916178 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.916183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.916185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.916295 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.916721 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.916725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.916728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.917000 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.917092 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.917930 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.918081 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260806 04:34:53.918483 23024 heartbeater.cc:347] Connected to a master server at 127.12.15.253:42907
I20260806 04:34:53.918820 23024 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:53.919582 23024 heartbeater.cc:511] Master 127.12.15.253:42907 requested a full tablet report, sending...
I20260806 04:34:53.921628 22458 ts_manager.cc:194] Registered new tserver with Master: abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159)
I20260806 04:34:53.922204 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.900725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.12.15.193:43937)
0806 04:34:53.901190 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.901241 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.902309 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.902828 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.902835 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.903274 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.903791 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.903809 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.904975 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.904979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.908715 (+ 3736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.908728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.909571 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.909575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.910143 (+ 568us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.910716 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.910733 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.913200 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.915712 (+ 2512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.915716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.915719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.916045 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.916459 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.916462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.916465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.916575 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.917273 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.917277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.917679 (+ 402us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.920305 (+ 2626us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.920518 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":158,"spinlock_wait_cycles":4480,"thread_start_us":102,"threads_started":1}
I20260806 04:34:53.923728 23022 heartbeater.cc:347] Connected to a master server at 127.12.15.252:38331
I20260806 04:34:53.923820 23022 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:53.923991 23022 heartbeater.cc:511] Master 127.12.15.252:38331 requested a full tablet report, sending...
Aug 06 04:34:53 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990893, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:53.930644 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.900273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:40651 (local address 127.12.15.254:34829)
0806 04:34:53.900447 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:53.900452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:53.901341 (+ 889us) server_negotiation.cc:407] Connection header received
0806 04:34:53.902540 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:53.902544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:53.902592 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:53.902644 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:53.903956 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.904805 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.925332 (+ 20527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.925567 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.925663 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.927604 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:53.927607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:53.927610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:53.927636 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:53.929098 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.929496 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.929500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.929502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.929590 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:53.929842 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:53.929845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:53.929847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:53.930041 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:53.930128 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:53.930378 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 04:34:53.930491 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 04:34:53.930666 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:53.900725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:40651)
0806 04:34:53.901226 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:53.901241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:53.902304 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:53.902830 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:53.902837 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:53.903312 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:53.903793 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.903808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.904944 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.904948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.925189 (+ 20241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:53.925201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:53.925676 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:53.925680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:53.925778 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:53.926245 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:53.926257 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:53.927453 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:53.929207 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.929210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.929212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.929407 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.929675 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:53.929678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:53.929680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:53.929760 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:53.930131 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:53.930134 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:53.930227 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 04:34:53.930315 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 04:34:53.930467 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":446,"spinlock_wait_cycles":49280,"thread_start_us":116,"threads_started":1}
I20260806 04:34:53.931670 23023 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34829
I20260806 04:34:53.931774 23023 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:53.931965 23023 heartbeater.cc:511] Master 127.12.15.254:34829 requested a full tablet report, sending...
I20260806 04:34:53.932986 21564 ts_manager.cc:194] Re-registered known tserver with Master: abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159)
I20260806 04:34:53.933403 22736 ts_manager.cc:194] Registered new tserver with Master: abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159)
I20260806 04:34:53.933880 21564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:40651
I20260806 04:34:53.939960 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:32835
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:34829
--webserver_port=38363
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:34:54.022389 22266 heartbeater.cc:503] Master 127.12.15.254:34829 was elected leader, sending a full tablet report...
W20260806 04:34:54.055274 23032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:54.055517 23032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:54.055557 23032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:54.059636 23032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:54.059693 23032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:54.059759 23032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:54.064951 23032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:32835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=38363
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:54.065989 23032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:54.066977 23032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:54.076558 23041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:54.076743 23032 server_base.cc:1034] running on GCE node
W20260806 04:34:54.076581 23043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:54.076609 23040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:54.077299 23032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:54.077898 23032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:54.079048 23032 hybrid_clock.cc:648] HybridClock initialized: now 1785990894079032 us; error 27 us; skew 500 ppm
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:54.081853 23032 init.cc:377] Logged in from keytab as oryx/127.12.15.194@KRBTEST.COM (short username oryx)
I20260806 04:34:54.082823 23032 webserver.cc:460] Webserver started at http://127.12.15.194:38363/ using document root <none> and password file <none>
I20260806 04:34:54.083382 23032 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:54.083461 23032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:54.087471 23032 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:54.089529 23062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:54.090792 23032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:54.090907 23032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "76cf6e444f59424880d1988ca591c7d5"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "893ae4624968bfedda518fbb363de2a1"
server_key_iv: "bb99d85b6c3693e2e80e76bb4a8c07b6"
server_key_version: "encryptionkey@0"
I20260806 04:34:54.091456 23032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:54.104732 23032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:54.107563 23032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:54.107730 23032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:54.108299 23032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:54.109184 23032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:54.109232 23032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:54.109283 23032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:54.109311 23032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:54.118074 23032 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:32835
I20260806 04:34:54.118146 23228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:32835 every 8 connection(s)
I20260806 04:34:54.119098 23032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:54.126971 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 23032
I20260806 04:34:54.127158 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 22064
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.252@KRBTEST.COM
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
I20260806 04:34:54.157553 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.121093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33645 (local address 127.12.15.252:38331)
0806 04:34:54.121236 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.121240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.122071 (+ 831us) server_negotiation.cc:407] Connection header received
0806 04:34:54.123226 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.123229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.123292 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.123367 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.124806 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.125426 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.149501 (+ 24075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.149955 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.150052 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.153353 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.153357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.153359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.153388 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.154800 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.155459 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.155463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.155465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.155559 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.155941 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.155945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.155947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.156204 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.156303 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.157136 (+ 833us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.157331 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:34:54.158627 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.121418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.12.15.194:33645)
0806 04:34:54.121910 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.121963 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.123025 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.123561 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.123568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.124030 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:54.124669 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.124683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.125573 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.125576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.149319 (+ 23743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.149342 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.150255 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.150259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.150542 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.151231 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:54.151252 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:54.153133 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:54.154946 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.154952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.154971 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.155345 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.155649 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.155653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.155655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.155830 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.156393 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:54.156400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:54.156775 (+ 375us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.157018 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.157408 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":80,"spinlock_wait_cycles":11008,"thread_start_us":111,"threads_started":1}
I20260806 04:34:54.160018 23229 heartbeater.cc:347] Connected to a master server at 127.12.15.252:38331
I20260806 04:34:54.160441 23229 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:54.161195 23229 heartbeater.cc:511] Master 127.12.15.252:38331 requested a full tablet report, sending...
I20260806 04:34:54.161558 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.122636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33789 (local address 127.12.15.253:42907)
0806 04:34:54.122772 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.122777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.122897 (+ 120us) server_negotiation.cc:407] Connection header received
0806 04:34:54.123269 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.123272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.123335 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.123388 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.124832 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.125651 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.126714 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.126985 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.127089 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.157913 (+ 30824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.157917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.157920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.157953 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.159620 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.160084 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.160088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.160090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.160187 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.160595 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.160598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.160600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.160814 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.160875 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.161214 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.161401 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:34:54.162420 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.122488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.12.15.194:33789)
0806 04:34:54.122820 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.122832 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.123060 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.123562 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.123570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.124041 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:54.124677 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.124686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.125792 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.125795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.126592 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.126601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.127195 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.127200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.155106 (+ 27906us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.156000 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:54.156015 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:54.157712 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:54.159747 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.159750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.159752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.159989 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.160279 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.160282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.160284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.160428 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.160942 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:54.160945 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:54.161037 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.161948 (+ 911us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.162109 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":139,"spinlock_wait_cycles":30976,"thread_start_us":80,"threads_started":1}
I20260806 04:34:54.162432 22736 ts_manager.cc:194] Registered new tserver with Master: 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
I20260806 04:34:54.163058 23232 heartbeater.cc:347] Connected to a master server at 127.12.15.253:42907
I20260806 04:34:54.163153 23232 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:54.163342 23232 heartbeater.cc:511] Master 127.12.15.253:42907 requested a full tablet report, sending...
I20260806 04:34:54.164057 22458 ts_manager.cc:194] Registered new tserver with Master: 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:54.168895 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:46261
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:34829
--webserver_port=43531
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:34:54.169929 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.121017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:40725 (local address 127.12.15.254:34829)
0806 04:34:54.121185 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.121190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.122083 (+ 893us) server_negotiation.cc:407] Connection header received
0806 04:34:54.123324 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.123327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.123378 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.123442 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.124828 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.125651 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.126878 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.127185 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.127274 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.166805 (+ 39531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.166810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.166812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.166841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.168236 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.168695 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.168698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.168700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.168783 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.169083 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.169086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.169088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.169317 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.169420 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.169659 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.169786 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 04:34:54.170132 23234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.121417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:40725)
0806 04:34:54.121911 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.121963 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.123053 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.123587 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.123590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.124039 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:54.124676 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.124686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.125758 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.125761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.126754 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.126767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.164725 (+ 37958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.164729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.164818 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.165249 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:54.165264 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:54.166641 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:54.168382 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.168385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.168387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.168599 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.168867 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.168870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.168872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.168950 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.169405 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:54.169407 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:54.169494 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.169574 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.169694 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":80,"spinlock_wait_cycles":49024,"thread_start_us":108,"threads_started":1}
I20260806 04:34:54.170848 23230 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34829
I20260806 04:34:54.170944 23230 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:54.171121 23230 heartbeater.cc:511] Master 127.12.15.254:34829 requested a full tablet report, sending...
I20260806 04:34:54.171799 21564 ts_manager.cc:194] Re-registered known tserver with Master: 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
I20260806 04:34:54.172433 21564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:40725
I20260806 04:34:54.246306 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.241560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34258)
0806 04:34:54.241839 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.241854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.241982 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.242782 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.242785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.242867 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:54.242987 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.242997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.244079 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.244082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.244948 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.244959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.245901 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.245905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.245999 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.246028 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.246099 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.246169 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":94,"threads_started":1}
I20260806 04:34:54.246337 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.241666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.12.15.252:38331)
0806 04:34:54.241815 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.241820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.242362 (+ 542us) server_negotiation.cc:407] Connection header received
0806 04:34:54.242462 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.242466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.242538 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.242639 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:54.243113 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.243951 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.245068 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.245705 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.245821 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.245981 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.246131 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.246207 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:34:54.247092 22760 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 04:34:54.247486 22810 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e456c908df943eca92a1ee10ae5a457. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:54.247612 22810 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: This master's current role is: LEARNER
W20260806 04:34:54.295501 23239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:54.295838 23239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:54.295897 23239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:54.299928 23239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:54.299984 23239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:54.300050 23239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:54.305670 23239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:46261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=43531
--tserver_master_addrs=127.12.15.254:34829,127.12.15.253:42907,127.12.15.252:38331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:54.306695 23239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:54.307708 23239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:54.317178 23249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:54.317193 23253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:54.317442 23239 server_base.cc:1034] running on GCE node
W20260806 04:34:54.317193 23250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:54.317924 23239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:54.318562 23239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:54.319713 23239 hybrid_clock.cc:648] HybridClock initialized: now 1785990894319689 us; error 43 us; skew 500 ppm
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:54.322335 23239 init.cc:377] Logged in from keytab as oryx/127.12.15.195@KRBTEST.COM (short username oryx)
I20260806 04:34:54.323156 23239 webserver.cc:460] Webserver started at http://127.12.15.195:43531/ using document root <none> and password file <none>
I20260806 04:34:54.323696 23239 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:54.323776 23239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:54.327615 23239 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:54.329704 23261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:54.330951 23239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:54.331071 23239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5859c5a9b5a64f5cb616be01df292198"
format_stamp: "Formatted at 2026-08-06 04:34:52 on dist-test-slave-xt4k"
server_key: "c11327f7699ed0b852a691dc739bb132"
server_key_iv: "cb1360e3e25230e57bd8cd84d6d2f69f"
server_key_version: "encryptionkey@0"
I20260806 04:34:54.331604 23239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:54.344167 23239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:54.347314 23239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:54.347523 23239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:54.348155 23239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:34:54.349172 23239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:54.349229 23239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:54.349289 23239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:54.349318 23239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:54.357986 23239 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:46261
I20260806 04:34:54.358028 23433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:46261 every 8 connection(s)
I20260806 04:34:54.359036 23239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:54.368628 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 23239
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.252@KRBTEST.COM
Aug 06 04:34:54 dist-test-slave-xt4k krb5kdc[21492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990894, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.253@KRBTEST.COM
I20260806 04:34:54.375432 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.362649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:43867 (local address 127.12.15.253:42907)
0806 04:34:54.362799 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.362804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.363591 (+ 787us) server_negotiation.cc:407] Connection header received
0806 04:34:54.363695 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.363697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.363739 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.363799 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.365049 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.365831 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367584 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.367901 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.368015 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.371962 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.371967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.371970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.371999 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.373604 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.374074 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.374077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.374079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.374164 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.374505 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.374508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.374510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.374723 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.374789 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.375134 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.375270 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:34:54.375964 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.361284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.12.15.195:34547)
0806 04:34:54.361999 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.362058 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.363075 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.363653 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.363663 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.364207 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:54.364876 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.364891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.365944 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.365947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.366959 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.366968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.367413 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.367653 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.368399 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:54.368414 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:54.370457 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:54.372661 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.372667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.372684 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.373058 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.373362 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.373365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.373367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.373535 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.374015 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:54.374022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:54.374325 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.374568 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.374891 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":105,"spinlock_wait_cycles":67968,"thread_start_us":272,"threads_started":1}
I20260806 04:34:54.375981 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.362537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.12.15.195:43867)
0806 04:34:54.363513 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.363526 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.363616 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.363833 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.363835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.364207 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:54.364884 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.364894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.366695 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.366698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.367472 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.367481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.368028 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.368032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.368122 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.370391 (+ 2269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:54.370405 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:54.371807 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:54.373731 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.373734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.373736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.373966 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.374271 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.374274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.374276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.374356 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.374861 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:54.374864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:54.374954 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.375682 (+ 728us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.375820 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":47,"spinlock_wait_cycles":102016,"thread_start_us":154,"threads_started":1}
I20260806 04:34:54.376796 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.360756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:34547 (local address 127.12.15.252:38331)
0806 04:34:54.360889 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.360905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.362162 (+ 1257us) server_negotiation.cc:407] Connection header received
0806 04:34:54.363314 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.363318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.363377 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.363462 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.365024 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.365831 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367067 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.367283 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367367 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.370641 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.370645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.370647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.370675 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.372538 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.373173 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.373177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.373178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.373261 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.373641 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.373644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.373646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.373855 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.373957 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.376538 (+ 2581us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.376652 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:34:54.377118 23436 heartbeater.cc:347] Connected to a master server at 127.12.15.253:42907
I20260806 04:34:54.377362 23436 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:54.377441 23434 heartbeater.cc:347] Connected to a master server at 127.12.15.252:38331
I20260806 04:34:54.377521 23434 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:54.377969 23436 heartbeater.cc:511] Master 127.12.15.253:42907 requested a full tablet report, sending...
I20260806 04:34:54.377969 23434 heartbeater.cc:511] Master 127.12.15.252:38331 requested a full tablet report, sending...
I20260806 04:34:54.379189 22736 ts_manager.cc:194] Registered new tserver with Master: 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261)
I20260806 04:34:54.379645 22458 ts_manager.cc:194] Registered new tserver with Master: 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261)
I20260806 04:34:54.380615 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.361284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:51173)
0806 04:34:54.361998 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.362058 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.363081 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.364048 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.364051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.364238 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:54.364886 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.364894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.365706 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.365708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.366971 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.366981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.367652 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.367720 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.368284 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:54.368328 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:54.371091 (+ 2763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:54.376982 (+ 5891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.376985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.376987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.377310 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.379183 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:54.379186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:54.379188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:54.379271 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:54.380065 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:54.380067 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:54.380157 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.380254 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.380404 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":134,"spinlock_wait_cycles":23552,"thread_start_us":344,"threads_started":1}
I20260806 04:34:54.380668 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.360828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:51173 (local address 127.12.15.254:34829)
0806 04:34:54.360938 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.360942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.362161 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 04:34:54.363314 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.363318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.363374 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.363462 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.365041 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.365564 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367102 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.367444 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.367515 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.371575 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.371579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.371582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.371611 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.373613 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.378965 (+ 5352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.378969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.378971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.379063 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.379375 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.379378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.379380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.379599 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.379773 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.380395 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.380517 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":2304}
I20260806 04:34:54.381386 23435 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34829
I20260806 04:34:54.381470 23435 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:54.381644 23435 heartbeater.cc:511] Master 127.12.15.254:34829 requested a full tablet report, sending...
I20260806 04:34:54.383188 21564 ts_manager.cc:194] Re-registered known tserver with Master: 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261)
I20260806 04:34:54.383882 21564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:51173
I20260806 04:34:54.384200 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.371421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.12.15.252:38331)
0806 04:34:54.372298 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.372302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.372647 (+ 345us) server_negotiation.cc:407] Connection header received
0806 04:34:54.372687 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.372690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.372736 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.372810 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.373846 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.374562 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.376020 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.377505 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.377600 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.380555 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.380559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.380561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.380584 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.382150 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.382692 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.382696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.382697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.382784 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.383084 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.383087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.383088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.383291 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.383404 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.383940 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.384054 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":67,"threads_started":1}
I20260806 04:34:54.386114 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.370617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.12.15.254:34829)
0806 04:34:54.370848 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.370853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.371463 (+ 610us) server_negotiation.cc:407] Connection header received
0806 04:34:54.371710 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.371713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.371760 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.373702 (+ 1942us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.374606 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.375375 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.376541 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.376859 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.376953 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.379527 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.379531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.379533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.379555 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.381644 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.382431 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.382435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.382437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.382583 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.382955 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.382959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.382960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.383163 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.383335 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.383760 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.383877 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260806 04:34:54.389258 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.371030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.12.15.253:42907)
0806 04:34:54.375612 (+ 4582us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.375616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.382060 (+ 6444us) server_negotiation.cc:407] Connection header received
0806 04:34:54.382231 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.382234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.382278 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.382327 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.383135 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.383817 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.384801 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.385062 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.385148 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.386294 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.386298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.386301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.386326 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.387800 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.388153 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.388156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.388157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.388239 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.388566 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.388569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.388570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.388732 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.388827 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.389046 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.389132 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4530,"thread_start_us":89,"threads_started":1}
I20260806 04:34:54.388741 21564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:54.391875 21564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:54.394567 22690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: NON_VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: true } } }
I20260806 04:34:54.394711 22690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:54.398072 23459 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: attempting to promote NON_VOTER ccc2dc9bff23409b807eb0ea5765e6dc to VOTER
W20260806 04:34:54.398331 22690 consensus_queue.cc:1159] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "329a4b2657dd4e629ea186a2946a12c6" responder_term: 1 status { last_received { term: 1 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 1 index: 8 } }
I20260806 04:34:54.398543 23459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } }
I20260806 04:34:54.399613 22474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Refusing update from remote peer 1e456c908df943eca92a1ee10ae5a457: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 04:34:54.399616 22760 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 LEARNER]: Refusing update from remote peer 1e456c908df943eca92a1ee10ae5a457: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 04:34:54.399940 23459 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 04:34:54.400101 23461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 04:34:54.401583 23459 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.401961 22474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.402704 22760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.403057 23459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.403138 23459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:54.403035 23460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.403237 23459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.403317 23459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:54.403276 23460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457 [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:54.403678 23474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.403810 23474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:54.404503 22810 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e456c908df943eca92a1ee10ae5a457" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:54.404605 22810 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:54.413697 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.408170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.12.15.195:46261)
0806 04:34:54.408592 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.408602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.409299 (+ 697us) server_negotiation.cc:407] Connection header received
0806 04:34:54.409377 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.409383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.409586 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.409972 (+ 386us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:54.410120 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.411158 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.412272 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.413177 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.413273 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.413425 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.413482 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.413567 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":109,"threads_started":1}
I20260806 04:34:54.413686 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.407463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:46261 (local address 127.0.0.1:37456)
0806 04:34:54.409207 (+ 1744us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.409216 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.409287 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.409739 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.409742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.409759 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:54.409836 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.409842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.411287 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.411290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.412155 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.412165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.413286 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.413289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.413373 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.413396 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.413456 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.413537 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1695,"mutex_wait_us":35,"thread_start_us":96,"threads_started":1}
I20260806 04:34:54.413925 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.407410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:32835 (local address 127.0.0.1:40156)
0806 04:34:54.408856 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.408867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.409015 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.409456 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.409459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.409478 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:54.409636 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.409645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.411223 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.411226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.412463 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.412470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.413518 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.413522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.413638 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.413659 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.413718 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.413789 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1391,"mutex_wait_us":80,"thread_start_us":128,"threads_started":1}
I20260806 04:34:54.414394 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.407013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36159 (local address 127.0.0.1:60622)
0806 04:34:54.407705 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.407718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.407780 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.409969 (+ 2189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.409972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.409987 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:54.410074 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.410081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.411177 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.411181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.412447 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.412459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.413518 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.413522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.414127 (+ 605us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.414150 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.414217 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.414286 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":497}
I20260806 04:34:54.414669 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.407325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.12.15.193:36159)
0806 04:34:54.409484 (+ 2159us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.409496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.409537 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:34:54.409612 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.409617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.409814 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.410223 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:54.410367 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.411071 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.412593 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.413380 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.413590 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.413737 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.414255 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.414356 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2061,"thread_start_us":109,"threads_started":1}
I20260806 04:34:54.414980 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.408041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.12.15.194:32835)
0806 04:34:54.408420 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.408434 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.409013 (+ 579us) server_negotiation.cc:407] Connection header received
0806 04:34:54.409104 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.409109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.409295 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.409795 (+ 500us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:54.409940 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.411127 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.412593 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.413380 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.414557 (+ 1177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.414699 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.414755 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.414846 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":130,"threads_started":1}
I20260806 04:34:54.417201 23330 tablet_service.cc:1467] Processing CreateTablet for tablet ba730c9810f3403aad7870553eba7dc8 (DEFAULT_TABLE table=test-table [id=f67be35835d24915a0d5c1c76aef7841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:54.417516 23131 tablet_service.cc:1467] Processing CreateTablet for tablet ba730c9810f3403aad7870553eba7dc8 (DEFAULT_TABLE table=test-table [id=f67be35835d24915a0d5c1c76aef7841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:54.417511 22919 tablet_service.cc:1467] Processing CreateTablet for tablet ba730c9810f3403aad7870553eba7dc8 (DEFAULT_TABLE table=test-table [id=f67be35835d24915a0d5c1c76aef7841]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:54.418210 23330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba730c9810f3403aad7870553eba7dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:54.418524 23131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba730c9810f3403aad7870553eba7dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:54.418524 22919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba730c9810f3403aad7870553eba7dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:54.425681 23482 tablet_bootstrap.cc:492] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Bootstrap starting.
I20260806 04:34:54.426877 23484 tablet_bootstrap.cc:492] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Bootstrap starting.
I20260806 04:34:54.427946 23482 tablet_bootstrap.cc:654] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:54.428640 23484 tablet_bootstrap.cc:654] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:54.428793 23482 log.cc:824] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:54.428694 23485 tablet_bootstrap.cc:492] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Bootstrap starting.
I20260806 04:34:54.429584 23484 log.cc:824] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:54.430842 23482 tablet_bootstrap.cc:492] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: No bootstrap required, opened a new log
I20260806 04:34:54.431035 23482 ts_tablet_manager.cc:1397] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:34:54.431246 23485 tablet_bootstrap.cc:654] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:54.431614 23484 tablet_bootstrap.cc:492] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: No bootstrap required, opened a new log
I20260806 04:34:54.431806 23484 ts_tablet_manager.cc:1397] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:34:54.432238 23485 log.cc:824] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:54.434162 23485 tablet_bootstrap.cc:492] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: No bootstrap required, opened a new log
I20260806 04:34:54.434329 23485 ts_tablet_manager.cc:1397] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:34:54.435056 23482 raft_consensus.cc:348] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.435304 23482 raft_consensus.cc:374] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:54.435359 23482 raft_consensus.cc:729] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5859c5a9b5a64f5cb616be01df292198, State: Initialized, Role: FOLLOWER
I20260806 04:34:54.435861 23482 consensus_queue.cc:260] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.435875 23484 raft_consensus.cc:348] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.436210 23484 raft_consensus.cc:374] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:54.436275 23484 raft_consensus.cc:729] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abaa6ef902644442aef07e3c40f04693, State: Initialized, Role: FOLLOWER
I20260806 04:34:54.436748 23484 consensus_queue.cc:260] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.436854 23482 ts_tablet_manager.cc:1428] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 04:34:54.437345 23435 heartbeater.cc:503] Master 127.12.15.254:34829 was elected leader, sending a full tablet report...
I20260806 04:34:54.437677 23484 ts_tablet_manager.cc:1428] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 04:34:54.441357 23023 heartbeater.cc:503] Master 127.12.15.254:34829 was elected leader, sending a full tablet report...
I20260806 04:34:54.441381 23485 raft_consensus.cc:348] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.441655 23485 raft_consensus.cc:374] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:54.441720 23485 raft_consensus.cc:729] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cf6e444f59424880d1988ca591c7d5, State: Initialized, Role: FOLLOWER
I20260806 04:34:54.442188 23485 consensus_queue.cc:260] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.443245 23485 ts_tablet_manager.cc:1428] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 04:34:54.444664 23230 heartbeater.cc:503] Master 127.12.15.254:34829 was elected leader, sending a full tablet report...
W20260806 04:34:54.610822 23437 tablet.cc:2369] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:34:54.620604 23233 tablet.cc:2369] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:34:54.661693 23025 tablet.cc:2369] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:54.675783 23493 raft_consensus.cc:482] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:54.675997 23493 raft_consensus.cc:504] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.677210 23493 leader_election.cc:288] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261), 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
I20260806 04:34:54.682618 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.677722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:39681 (local address 127.12.15.194:32835)
0806 04:34:54.677870 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.677875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.677950 (+ 75us) server_negotiation.cc:407] Connection header received
0806 04:34:54.678124 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.678127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.678181 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.678272 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:54.678749 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.679546 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.681347 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.681973 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.682108 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.682165 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.682419 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.682491 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:34:54.682618 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.677600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49923 (local address 127.12.15.195:46261)
0806 04:34:54.677870 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.677875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.677936 (+ 61us) server_negotiation.cc:407] Connection header received
0806 04:34:54.678124 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.678127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.678181 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.678278 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:54.678749 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.679546 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.681347 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.681973 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.682107 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.682165 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.682379 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.682480 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260806 04:34:54.682680 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.677511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:46261 (local address 127.12.15.193:49923)
0806 04:34:54.677842 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.677859 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.678005 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.678348 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.678351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.678383 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:54.678614 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.678622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.679663 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.679667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.681194 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.681207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.682126 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.682130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.682231 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.682261 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.682345 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.682441 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":119,"spinlock_wait_cycles":20480,"thread_start_us":120,"threads_started":1}
I20260806 04:34:54.682722 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.677535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:32835 (local address 127.12.15.193:39681)
0806 04:34:54.677862 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:54.677874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:54.678005 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:54.678348 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:54.678351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:54.678383 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:54.678625 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.678632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.679663 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.679667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.681194 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:54.681207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.682213 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.682216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:54.682293 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.682311 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 04:34:54.682369 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 04:34:54.682440 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":67,"spinlock_wait_cycles":8704,"thread_start_us":113,"threads_started":1}
I20260806 04:34:54.683640 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba730c9810f3403aad7870553eba7dc8" candidate_uuid: "abaa6ef902644442aef07e3c40f04693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cf6e444f59424880d1988ca591c7d5" is_pre_election: true
I20260806 04:34:54.683694 23360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba730c9810f3403aad7870553eba7dc8" candidate_uuid: "abaa6ef902644442aef07e3c40f04693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5859c5a9b5a64f5cb616be01df292198" is_pre_election: true
I20260806 04:34:54.683965 23155 raft_consensus.cc:2454] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abaa6ef902644442aef07e3c40f04693 in term 0.
I20260806 04:34:54.683969 23360 raft_consensus.cc:2454] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abaa6ef902644442aef07e3c40f04693 in term 0.
I20260806 04:34:54.684482 22847 leader_election.cc:302] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cf6e444f59424880d1988ca591c7d5, abaa6ef902644442aef07e3c40f04693; no voters:
I20260806 04:34:54.684762 23493 raft_consensus.cc:2781] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:54.684852 23493 raft_consensus.cc:482] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:54.684901 23493 raft_consensus.cc:3037] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:54.686129 23493 raft_consensus.cc:504] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.686466 23493 leader_election.cc:288] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [CANDIDATE]: Term 1 election: Requested vote from peers 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261), 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
I20260806 04:34:54.686827 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba730c9810f3403aad7870553eba7dc8" candidate_uuid: "abaa6ef902644442aef07e3c40f04693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cf6e444f59424880d1988ca591c7d5"
I20260806 04:34:54.686944 23155 raft_consensus.cc:3037] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:54.687198 23360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba730c9810f3403aad7870553eba7dc8" candidate_uuid: "abaa6ef902644442aef07e3c40f04693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5859c5a9b5a64f5cb616be01df292198"
I20260806 04:34:54.687312 23360 raft_consensus.cc:3037] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:54.687994 23155 raft_consensus.cc:2454] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abaa6ef902644442aef07e3c40f04693 in term 1.
I20260806 04:34:54.688282 22847 leader_election.cc:302] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76cf6e444f59424880d1988ca591c7d5, abaa6ef902644442aef07e3c40f04693; no voters:
I20260806 04:34:54.688462 23493 raft_consensus.cc:2781] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:54.688653 23360 raft_consensus.cc:2454] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abaa6ef902644442aef07e3c40f04693 in term 1.
I20260806 04:34:54.688675 23493 raft_consensus.cc:686] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 1 LEADER]: Becoming Leader. State: Replica: abaa6ef902644442aef07e3c40f04693, State: Running, Role: LEADER
I20260806 04:34:54.689134 23493 consensus_queue.cc:237] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:54.692605 21560 catalog_manager.cc:5495] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 reported cstate change: term changed from 0 to 1, leader changed from <none> to abaa6ef902644442aef07e3c40f04693 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "abaa6ef902644442aef07e3c40f04693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:54.706828 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.703309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.12.15.193:36159)
0806 04:34:54.703433 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.703437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.703450 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:54.703554 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.703556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.703616 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.703705 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:54.703929 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.704497 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.705368 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.705730 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.705815 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.706089 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:54.706541 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:54.706637 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.706666 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.706716 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:34:54.713032 23155 raft_consensus.cc:1261] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Refusing update from remote peer abaa6ef902644442aef07e3c40f04693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:54.713034 23360 raft_consensus.cc:1261] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Refusing update from remote peer abaa6ef902644442aef07e3c40f04693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:54.713810 23493 consensus_queue.cc:1035] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:54.714035 23498 consensus_queue.cc:1035] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:54.726200 23503 mvcc.cc:205] Tried to move back new op lower bound from 7315418704737021952 to 7315418704649334784. Current Snapshot: MvccSnapshot[applied={T|T < 7315418704737021952}]
I20260806 04:34:54.733425 21560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:54.733594 21560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:54.738281 21560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e456c908df943eca92a1ee10ae5a457: Sending DeleteTablet for 3 replicas of tablet ba730c9810f3403aad7870553eba7dc8
I20260806 04:34:54.739022 23330 tablet_service.cc:1514] Processing DeleteTablet for tablet ba730c9810f3403aad7870553eba7dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:54 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:37456
I20260806 04:34:54.739020 22919 tablet_service.cc:1514] Processing DeleteTablet for tablet ba730c9810f3403aad7870553eba7dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:54 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:60622
I20260806 04:34:54.739341 23131 tablet_service.cc:1514] Processing DeleteTablet for tablet ba730c9810f3403aad7870553eba7dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:54 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:40156
I20260806 04:34:54.739567 23517 tablet_replica.cc:330] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: stopping tablet replica
I20260806 04:34:54.739765 23520 tablet_replica.cc:330] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: stopping tablet replica
I20260806 04:34:54.739889 23517 raft_consensus.cc:2229] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:54.740155 23520 raft_consensus.cc:2229] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:54.740309 23517 raft_consensus.cc:2258] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:54.740636 23520 raft_consensus.cc:2258] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:54.742583 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 21510
I20260806 04:34:54.750488 23520 ts_tablet_manager.cc:1905] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:54.752352 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.741452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.12.15.253:42907)
0806 04:34:54.741559 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.741563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.741576 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:54.741614 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.741617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.741663 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.741746 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.742773 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.743431 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.746115 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.746466 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.746540 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.748689 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.748693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.748696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.748725 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.750406 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.750884 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.750888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.750889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.750974 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.751356 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.751360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.751362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.751585 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.751686 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.752026 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.752164 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:34:54.753849 23520 ts_tablet_manager.cc:1918] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:54.754019 23520 log.cc:1196] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/ba730c9810f3403aad7870553eba7dc8
I20260806 04:34:54.754397 23520 ts_tablet_manager.cc:1939] T ba730c9810f3403aad7870553eba7dc8 P 76cf6e444f59424880d1988ca591c7d5: Deleting consensus metadata
W20260806 04:34:54.759712 23067 connection.cc:535] server connection from 127.0.0.1:40156 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 04:34:54.761374 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.741046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.12.15.252:38331)
0806 04:34:54.741159 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.741164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.752891 (+ 11727us) server_negotiation.cc:407] Connection header received
0806 04:34:54.753064 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.753068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.753128 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.753177 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.753971 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.754774 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.755889 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.756223 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.756315 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.758061 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.758064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.758067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.758092 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.759685 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.760129 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.760132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.760134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.760227 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.760515 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.760518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.760520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.760725 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.760856 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.761098 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.761227 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:34:54.764441 23517 ts_tablet_manager.cc:1905] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:54.767499 23517 ts_tablet_manager.cc:1918] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:54.767640 23517 log.cc:1196] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/ba730c9810f3403aad7870553eba7dc8
I20260806 04:34:54.767961 23517 ts_tablet_manager.cc:1939] T ba730c9810f3403aad7870553eba7dc8 P abaa6ef902644442aef07e3c40f04693: Deleting consensus metadata
W20260806 04:34:54.769140 22848 connection.cc:441] server connection from 127.0.0.1:60622 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60622 (request call id 1) could send its response
I20260806 04:34:54.769444 23516 tablet_replica.cc:330] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: stopping tablet replica
I20260806 04:34:54.769779 23516 raft_consensus.cc:2229] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:54.770134 23516 raft_consensus.cc:2258] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:54.771616 23516 ts_tablet_manager.cc:1905] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:54.774420 23516 ts_tablet_manager.cc:1918] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:54.774595 23516 log.cc:1196] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/ba730c9810f3403aad7870553eba7dc8
I20260806 04:34:54.774955 23516 ts_tablet_manager.cc:1939] T ba730c9810f3403aad7870553eba7dc8 P 5859c5a9b5a64f5cb616be01df292198: Deleting consensus metadata
W20260806 04:34:54.776122 23265 connection.cc:441] server connection from 127.0.0.1:37456 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37456 (request call id 1) could send its response
I20260806 04:34:54.781261 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.771600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.12.15.253:42907)
0806 04:34:54.771704 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.771707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.772353 (+ 646us) server_negotiation.cc:407] Connection header received
0806 04:34:54.772520 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.772523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.772568 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.772611 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.773572 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.774335 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.775725 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.776077 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.776157 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.777963 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.777967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.777970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.777997 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.779510 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.779909 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.779913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.779915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.780001 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.780284 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.780287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.780289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.780527 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.780627 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.780995 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.781105 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 04:34:54.783069 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:54.774713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.12.15.252:38331)
0806 04:34:54.774823 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:54.774827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:54.775030 (+ 203us) server_negotiation.cc:407] Connection header received
0806 04:34:54.775179 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:54.775182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:54.775221 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:54.775282 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:54.776315 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.777056 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.778120 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:54.778392 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:54.778490 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:54.779998 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:54.780001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:54.780004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:54.780027 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:54.781515 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.781912 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.781915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.781917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.782007 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:54.782408 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:54.782410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:54.782412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:54.782578 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:54.782667 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:54.782876 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 04:34:54.782961 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:34:55.727386 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:55.726863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:45389)
0806 04:34:55.727173 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:55.727274 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":79,"threads_started":1}
I20260806 04:34:55.727386 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:55.726864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:60219)
0806 04:34:55.727208 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:55.727281 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":86,"threads_started":1}
W20260806 04:34:55.727788 23023 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:34829 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
W20260806 04:34:55.727914 23435 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:34829 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
I20260806 04:34:55.728068 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:55.727596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:33863)
0806 04:34:55.727865 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:55.727965 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":65,"threads_started":1}
W20260806 04:34:55.728389 23230 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:34829 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
I20260806 04:34:56.238516 23533 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1e456c908df943eca92a1ee10ae5a457)
I20260806 04:34:56.238751 23533 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } }
I20260806 04:34:56.239235 23535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1e456c908df943eca92a1ee10ae5a457)
I20260806 04:34:56.239392 23535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } }
I20260806 04:34:56.239830 23533 leader_election.cc:288] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829), 329a4b2657dd4e629ea186a2946a12c6 (127.12.15.253:42907)
I20260806 04:34:56.240759 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.240048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50052)
0806 04:34:56.240565 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.240653 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":94,"threads_started":1}
I20260806 04:34:56.240881 23535 leader_election.cc:288] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829), ccc2dc9bff23409b807eb0ea5765e6dc (127.12.15.252:38331)
I20260806 04:34:56.241302 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.240560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50054)
0806 04:34:56.240799 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.240873 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
W20260806 04:34:56.241456 22453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
W20260806 04:34:56.241457 22729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
I20260806 04:34:56.242424 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.242165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50068)
0806 04:34:56.242300 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.242350 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 04:34:56.242581 22453 leader_election.cc:334] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829): Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
I20260806 04:34:56.242697 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.242388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50076)
0806 04:34:56.242575 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.242622 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260806 04:34:56.242805 22729 leader_election.cc:334] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829): Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
I20260806 04:34:56.244639 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.240449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42907 (local address 127.0.0.1:51276)
0806 04:34:56.241002 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.241015 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.241124 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.241370 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.241373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.241393 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.241503 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.241510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.242487 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.242491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.243556 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.243568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.244218 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.244222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.244312 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.244361 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.244438 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.244511 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":39,"thread_start_us":101,"threads_started":1}
I20260806 04:34:56.244642 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.240783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.12.15.253:42907)
0806 04:34:56.241107 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.241111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.241126 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:34:56.241214 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.241217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.241260 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.241340 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.241636 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.242365 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.243661 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.244112 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.244191 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.244241 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.244462 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.244533 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":51,"threads_started":1}
I20260806 04:34:56.245237 22474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "329a4b2657dd4e629ea186a2946a12c6" is_pre_election: true
I20260806 04:34:56.245414 22474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccc2dc9bff23409b807eb0ea5765e6dc in term 1.
I20260806 04:34:56.245689 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.240577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:38331 (local address 127.0.0.1:34838)
0806 04:34:56.240998 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.241012 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.241124 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.241821 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.241824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.241840 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.241936 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.241944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.243507 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.243538 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.244483 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.244493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.245309 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.245313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.245399 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.245420 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.245485 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.245556 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":57,"threads_started":1}
I20260806 04:34:56.245689 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.240978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.12.15.252:38331)
0806 04:34:56.241543 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.241547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.241559 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:34:56.241600 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.241602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.241642 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.241722 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.242058 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.243397 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.244620 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.245201 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.245295 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.245349 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.245505 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.245579 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":500,"thread_start_us":72,"threads_started":1}
I20260806 04:34:56.245769 22729 leader_election.cc:302] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 329a4b2657dd4e629ea186a2946a12c6, ccc2dc9bff23409b807eb0ea5765e6dc; no voters: 1e456c908df943eca92a1ee10ae5a457
I20260806 04:34:56.245993 23533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 04:34:56.246054 23533 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1e456c908df943eca92a1ee10ae5a457)
I20260806 04:34:56.246080 23533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:34:56.246116 22760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "329a4b2657dd4e629ea186a2946a12c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" is_pre_election: true
I20260806 04:34:56.247036 23533 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } }
I20260806 04:34:56.247248 22760 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 329a4b2657dd4e629ea186a2946a12c6 in current term 2: Already voted for candidate ccc2dc9bff23409b807eb0ea5765e6dc in this term.
I20260806 04:34:56.247354 23533 leader_election.cc:288] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [CANDIDATE]: Term 2 election: Requested vote from peers 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829), 329a4b2657dd4e629ea186a2946a12c6 (127.12.15.253:42907)
I20260806 04:34:56.247746 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.247476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50088)
0806 04:34:56.247592 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.247687 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 04:34:56.248066 22450 leader_election.cc:302] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 329a4b2657dd4e629ea186a2946a12c6; no voters: 1e456c908df943eca92a1ee10ae5a457, ccc2dc9bff23409b807eb0ea5765e6dc
I20260806 04:34:56.248262 23535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:34:56.248266 22474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "329a4b2657dd4e629ea186a2946a12c6"
I20260806 04:34:56.248993 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.248801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50094)
0806 04:34:56.248899 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.248945 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 04:34:56.249089 22729 leader_election.cc:334] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829): Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
I20260806 04:34:56.249322 23535 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:34:56.250211 22474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccc2dc9bff23409b807eb0ea5765e6dc in term 2.
I20260806 04:34:56.250723 22729 leader_election.cc:302] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 329a4b2657dd4e629ea186a2946a12c6, ccc2dc9bff23409b807eb0ea5765e6dc; no voters: 1e456c908df943eca92a1ee10ae5a457
I20260806 04:34:56.250886 23533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 04:34:56.251089 23533 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [term 2 LEADER]: Becoming Leader. State: Replica: ccc2dc9bff23409b807eb0ea5765e6dc, State: Running, Role: LEADER
I20260806 04:34:56.251318 23533 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } }
I20260806 04:34:56.252578 23547 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccc2dc9bff23409b807eb0ea5765e6dc. Latest consensus state: current_term: 2 leader_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:56.252701 23547 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:56.253010 23550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:56.255365 23550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:56.255753 23550 catalog_manager.cc:1259] Loaded cluster ID: 96112cd7fa4446f6880d6d83cfb22b7a
I20260806 04:34:56.255797 23550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:56.256354 23550 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:56.256736 23550 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Loaded TSK: 0
I20260806 04:34:56.257408 23550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:34:56.305480 22736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:56.308660 22736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:56.311661 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.311401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50124)
0806 04:34:56.311514 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.311585 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 04:34:56.311965 22474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [term 2 FOLLOWER]: Refusing update from remote peer ccc2dc9bff23409b807eb0ea5765e6dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 04:34:56.312407 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.312164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.0.0.1:50136)
0806 04:34:56.312260 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.312318 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 04:34:56.312546 22729 consensus_peers.cc:487] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc -> Peer 1e456c908df943eca92a1ee10ae5a457 (127.12.15.254:34829): Couldn't send request to peer 1e456c908df943eca92a1ee10ae5a457. Status: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 04:34:56.312695 23533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 04:34:56.314462 23535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccc2dc9bff23409b807eb0ea5765e6dc. Latest consensus state: current_term: 2 leader_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:56.314601 23535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:56.315824 23533 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:56.315968 23533 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:56.316469 23535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1e456c908df943eca92a1ee10ae5a457" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34829 } } peers { permanent_uuid: "329a4b2657dd4e629ea186a2946a12c6" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "ccc2dc9bff23409b807eb0ea5765e6dc" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 38331 } attrs { promote: false } } }
I20260806 04:34:56.316725 23535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 329a4b2657dd4e629ea186a2946a12c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:34:56.326745 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.322495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36159 (local address 127.0.0.1:34290)
0806 04:34:56.322667 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.322682 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.322762 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.323098 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.323101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.323122 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.323232 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.323239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.324224 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.324228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.325429 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.325440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.326248 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.326252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.326339 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.326369 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.326447 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.326521 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 04:34:56.326850 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.322623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34290 (local address 127.12.15.193:36159)
0806 04:34:56.322879 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.322883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.322897 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:56.322940 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.322943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.322985 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.323072 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.323378 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.324111 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.325560 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.326145 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.326231 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.326287 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.326620 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.326716 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260806 04:34:56.327747 22919 tablet_service.cc:1467] Processing CreateTablet for tablet 984c61af04c24bb38b48c0b7f47d2ac3 (DEFAULT_TABLE table=test-table [id=6d7df04a853d447e8c8d5a17ed0f260c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:56.327868 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.322954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:46261 (local address 127.0.0.1:52464)
0806 04:34:56.323321 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.323336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.323422 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.323654 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.323657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.323674 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.323764 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.323771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.324680 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.324684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.325727 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.325734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.327455 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.327458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.327581 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.327605 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.327681 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.327756 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297}
I20260806 04:34:56.327970 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.323093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.12.15.195:46261)
0806 04:34:56.323324 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.323329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.323384 (+ 55us) server_negotiation.cc:407] Connection header received
0806 04:34:56.323500 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.323503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.323544 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.323620 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.323866 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.324542 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.325836 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.327229 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.327330 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.327393 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.327745 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.327826 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260806 04:34:56.328125 22919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 984c61af04c24bb38b48c0b7f47d2ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:56.328526 23330 tablet_service.cc:1467] Processing CreateTablet for tablet 984c61af04c24bb38b48c0b7f47d2ac3 (DEFAULT_TABLE table=test-table [id=6d7df04a853d447e8c8d5a17ed0f260c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:56.328917 23330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 984c61af04c24bb38b48c0b7f47d2ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:56.330451 23563 tablet_bootstrap.cc:492] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: Bootstrap starting.
I20260806 04:34:56.331348 23565 tablet_bootstrap.cc:492] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Bootstrap starting.
I20260806 04:34:56.331849 23563 tablet_bootstrap.cc:654] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:56.331997 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.324566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.12.15.194:32835)
0806 04:34:56.324801 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.324805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.324818 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:56.324867 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.324869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.324907 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.324982 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.325239 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.325861 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.330662 (+ 4801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.331297 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.331411 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.331611 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.331772 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.331862 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260806 04:34:56.331977 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.324170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:32835 (local address 127.0.0.1:52260)
0806 04:34:56.324447 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.324460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.324557 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.325006 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.325009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.325031 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.325133 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.325139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.325951 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.325954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.327421 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.327431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.331531 (+ 4100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.331534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.331639 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.331665 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.331748 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.331837 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260806 04:34:56.332592 23131 tablet_service.cc:1467] Processing CreateTablet for tablet 984c61af04c24bb38b48c0b7f47d2ac3 (DEFAULT_TABLE table=test-table [id=6d7df04a853d447e8c8d5a17ed0f260c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:56.332850 23131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 984c61af04c24bb38b48c0b7f47d2ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:56.333456 23563 tablet_bootstrap.cc:492] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: No bootstrap required, opened a new log
I20260806 04:34:56.333544 23563 ts_tablet_manager.cc:1397] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:56.333981 23563 raft_consensus.cc:348] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.334084 23563 raft_consensus.cc:374] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:56.334115 23563 raft_consensus.cc:729] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abaa6ef902644442aef07e3c40f04693, State: Initialized, Role: FOLLOWER
I20260806 04:34:56.334249 23563 consensus_queue.cc:260] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.334630 23563 ts_tablet_manager.cc:1428] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:56.335166 23571 tablet_bootstrap.cc:492] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Bootstrap starting.
I20260806 04:34:56.335983 23571 tablet_bootstrap.cc:654] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:56.336647 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.334915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:47267)
0806 04:34:56.336512 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.336566 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1527,"thread_start_us":103,"threads_started":1}
I20260806 04:34:56.337237 23571 tablet_bootstrap.cc:492] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: No bootstrap required, opened a new log
I20260806 04:34:56.337333 23571 ts_tablet_manager.cc:1397] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:56.337353 23022 heartbeater.cc:503] Master 127.12.15.252:38331 was elected leader, sending a full tablet report...
I20260806 04:34:56.337796 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.337136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:59211)
0806 04:34:56.337679 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.337731 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479}
I20260806 04:34:56.337766 23571 raft_consensus.cc:348] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.337865 23571 raft_consensus.cc:374] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:56.337896 23571 raft_consensus.cc:729] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cf6e444f59424880d1988ca591c7d5, State: Initialized, Role: FOLLOWER
I20260806 04:34:56.338054 23571 consensus_queue.cc:260] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.338383 23571 ts_tablet_manager.cc:1428] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:56.339222 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.338743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:45571)
0806 04:34:56.339069 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.339124 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":95,"threads_started":1}
I20260806 04:34:56.339715 23565 tablet_bootstrap.cc:654] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:56.339965 23229 heartbeater.cc:503] Master 127.12.15.252:38331 was elected leader, sending a full tablet report...
I20260806 04:34:56.340421 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.339746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:33093)
0806 04:34:56.340276 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.340326 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464}
I20260806 04:34:56.341081 23565 tablet_bootstrap.cc:492] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: No bootstrap required, opened a new log
I20260806 04:34:56.341178 23565 ts_tablet_manager.cc:1397] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 04:34:56.341552 23565 raft_consensus.cc:348] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.341650 23565 raft_consensus.cc:374] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:56.341681 23565 raft_consensus.cc:729] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5859c5a9b5a64f5cb616be01df292198, State: Initialized, Role: FOLLOWER
I20260806 04:34:56.341804 23565 consensus_queue.cc:260] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.342080 23565 ts_tablet_manager.cc:1428] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:56.342777 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.342424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:43659)
0806 04:34:56.342656 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.342705 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260806 04:34:56.343601 23434 heartbeater.cc:503] Master 127.12.15.252:38331 was elected leader, sending a full tablet report...
I20260806 04:34:56.343827 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.343229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:51755)
0806 04:34:56.343723 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.343772 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421}
I20260806 04:34:56.358172 23579 raft_consensus.cc:482] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:56.358299 23579 raft_consensus.cc:504] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.359519 23579 leader_election.cc:288] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159), 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
I20260806 04:34:56.363724 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.359760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:32835 (local address 127.12.15.195:37785)
0806 04:34:56.360316 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.360326 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.360444 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.360784 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.360787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.360803 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.360885 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.360889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.361589 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.361593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.362495 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.362505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.363217 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.363222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.363351 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.363377 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.363487 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.363561 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":35,"thread_start_us":126,"threads_started":1}
I20260806 04:34:56.363734 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.360097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:37785 (local address 127.12.15.194:32835)
0806 04:34:56.360221 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.360226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.360414 (+ 188us) server_negotiation.cc:407] Connection header received
0806 04:34:56.360563 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.360566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.360614 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.360660 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.360987 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.361490 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.362620 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.363009 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.363114 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.363174 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.363519 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.363615 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:34:56.364570 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.360133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:52529 (local address 127.12.15.193:36159)
0806 04:34:56.360244 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.360248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.360261 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:56.360313 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.360316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.360399 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.360474 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:56.360747 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.361524 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.363278 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.363861 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.363966 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.364013 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.364372 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.364455 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:34:56.364641 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984c61af04c24bb38b48c0b7f47d2ac3" candidate_uuid: "5859c5a9b5a64f5cb616be01df292198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cf6e444f59424880d1988ca591c7d5" is_pre_election: true
I20260806 04:34:56.364810 23155 raft_consensus.cc:2454] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5859c5a9b5a64f5cb616be01df292198 in term 0.
I20260806 04:34:56.365012 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.359698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36159 (local address 127.12.15.195:52529)
0806 04:34:56.360054 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.360070 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:56.360172 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:56.360494 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:56.360497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:56.360513 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:56.360653 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.360659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.361655 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.361659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.363124 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:56.363134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.364071 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.364074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:56.364172 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.364195 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:34:56.364267 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 04:34:56.364887 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":20}
I20260806 04:34:56.365300 23263 leader_election.cc:302] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5859c5a9b5a64f5cb616be01df292198, 76cf6e444f59424880d1988ca591c7d5; no voters:
I20260806 04:34:56.365492 22947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984c61af04c24bb38b48c0b7f47d2ac3" candidate_uuid: "5859c5a9b5a64f5cb616be01df292198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abaa6ef902644442aef07e3c40f04693" is_pre_election: true
I20260806 04:34:56.365593 23579 raft_consensus.cc:2781] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:56.365682 23579 raft_consensus.cc:482] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:56.365721 23579 raft_consensus.cc:3037] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:56.365756 22947 raft_consensus.cc:2454] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5859c5a9b5a64f5cb616be01df292198 in term 0.
I20260806 04:34:56.366925 23579 raft_consensus.cc:504] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.367244 23579 leader_election.cc:288] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [CANDIDATE]: Term 1 election: Requested vote from peers abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159), 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835)
I20260806 04:34:56.367568 22947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984c61af04c24bb38b48c0b7f47d2ac3" candidate_uuid: "5859c5a9b5a64f5cb616be01df292198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abaa6ef902644442aef07e3c40f04693"
I20260806 04:34:56.367611 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "984c61af04c24bb38b48c0b7f47d2ac3" candidate_uuid: "5859c5a9b5a64f5cb616be01df292198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76cf6e444f59424880d1988ca591c7d5"
I20260806 04:34:56.367697 22947 raft_consensus.cc:3037] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:56.367722 23155 raft_consensus.cc:3037] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:56.368846 23155 raft_consensus.cc:2454] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5859c5a9b5a64f5cb616be01df292198 in term 1.
I20260806 04:34:56.368846 22947 raft_consensus.cc:2454] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5859c5a9b5a64f5cb616be01df292198 in term 1.
I20260806 04:34:56.369108 23265 leader_election.cc:302] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5859c5a9b5a64f5cb616be01df292198, abaa6ef902644442aef07e3c40f04693; no voters:
I20260806 04:34:56.369259 23579 raft_consensus.cc:2781] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:56.369491 23579 raft_consensus.cc:686] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 1 LEADER]: Becoming Leader. State: Replica: 5859c5a9b5a64f5cb616be01df292198, State: Running, Role: LEADER
I20260806 04:34:56.369750 23579 consensus_queue.cc:237] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } }
I20260806 04:34:56.370160 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.369946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:57921)
0806 04:34:56.370049 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.370102 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 04:34:56.372330 22744 catalog_manager.cc:5495] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195). New cstate: current_term: 1 leader_uuid: "5859c5a9b5a64f5cb616be01df292198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5859c5a9b5a64f5cb616be01df292198" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 46261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:56.396716 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.393390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52472 (local address 127.12.15.195:46261)
0806 04:34:56.393542 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:56.393546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:56.393559 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:56.393634 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:56.393636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:56.393689 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:56.393780 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:56.394001 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.394537 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.395336 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:56.395560 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:56.395638 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:56.395860 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:56.396389 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:56.396487 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:56.396543 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 04:34:56.396596 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 04:34:56.399521 23155 raft_consensus.cc:1261] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Refusing update from remote peer 5859c5a9b5a64f5cb616be01df292198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:56.399789 22947 raft_consensus.cc:1261] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Refusing update from remote peer 5859c5a9b5a64f5cb616be01df292198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:56.400035 23582 consensus_queue.cc:1035] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76cf6e444f59424880d1988ca591c7d5" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:56.400393 23579 consensus_queue.cc:1035] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abaa6ef902644442aef07e3c40f04693" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:56.402237 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.401525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:55539)
0806 04:34:56.402091 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.402158 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491}
I20260806 04:34:56.402560 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.402059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:38465)
0806 04:34:56.402424 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.402479 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280}
I20260806 04:34:56.406149 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.405465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:44115)
0806 04:34:56.406027 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.406083 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483}
I20260806 04:34:56.407274 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.406824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.195:55983)
0806 04:34:56.407155 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.407208 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274}
I20260806 04:34:56.407577 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.406867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.193:48517)
0806 04:34:56.407446 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.407500 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501}
I20260806 04:34:56.407771 23593 mvcc.cc:205] Tried to move back new op lower bound from 7315418711647711232 to 7315418711533268992. Current Snapshot: MvccSnapshot[applied={T|T < 7315418711647711232 or (T in {7315418711647711232})}]
I20260806 04:34:56.409065 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:56.408828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34829 (local address 127.12.15.194:50103)
0806 04:34:56.408935 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.408988 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:34829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 04:34:56.414588 22736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:56.414762 22736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:56.418694 22736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccc2dc9bff23409b807eb0ea5765e6dc: Sending DeleteTablet for 3 replicas of tablet 984c61af04c24bb38b48c0b7f47d2ac3
I20260806 04:34:56.419343 23330 tablet_service.cc:1514] Processing DeleteTablet for tablet 984c61af04c24bb38b48c0b7f47d2ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:56 UTC) from {username='oryx', principal='oryx/127.12.15.252@KRBTEST.COM'} at 127.0.0.1:52464
I20260806 04:34:56.419488 22919 tablet_service.cc:1514] Processing DeleteTablet for tablet 984c61af04c24bb38b48c0b7f47d2ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:56 UTC) from {username='oryx', principal='oryx/127.12.15.252@KRBTEST.COM'} at 127.0.0.1:34290
I20260806 04:34:56.419759 23602 tablet_replica.cc:330] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: stopping tablet replica
I20260806 04:34:56.419940 23602 raft_consensus.cc:2229] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:56.420107 23603 tablet_replica.cc:330] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: stopping tablet replica
I20260806 04:34:56.420109 23131 tablet_service.cc:1514] Processing DeleteTablet for tablet 984c61af04c24bb38b48c0b7f47d2ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:56 UTC) from {username='oryx', principal='oryx/127.12.15.252@KRBTEST.COM'} at 127.0.0.1:52260
I20260806 04:34:56.420233 23603 raft_consensus.cc:2229] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:56.420436 23603 raft_consensus.cc:2258] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:56.420444 23602 raft_consensus.cc:2258] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:56.421192 23603 ts_tablet_manager.cc:1905] T 984c61af04c24bb38b48c0b7f47d2ac3 P abaa6ef902644442aef07e3c40f04693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:56.421422 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 22688
I20260806 04:34:56.421444 23602 ts_tablet_manager.cc:1905] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:56.428448 23602 ts_tablet_manager.cc:1918] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:56.428548 23602 log.cc:1196] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/984c61af04c24bb38b48c0b7f47d2ac3
I20260806 04:34:56.428785 23602 ts_tablet_manager.cc:1939] T 984c61af04c24bb38b48c0b7f47d2ac3 P 5859c5a9b5a64f5cb616be01df292198: Deleting consensus metadata
I20260806 04:34:56.429075 23604 tablet_replica.cc:330] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: stopping tablet replica
I20260806 04:34:56.429193 23604 raft_consensus.cc:2229] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:56.429332 23604 raft_consensus.cc:2258] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:56.442809 23604 ts_tablet_manager.cc:1905] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:56.446099 23604 ts_tablet_manager.cc:1918] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:56.446179 23604 log.cc:1196] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/984c61af04c24bb38b48c0b7f47d2ac3
I20260806 04:34:56.446399 23604 ts_tablet_manager.cc:1939] T 984c61af04c24bb38b48c0b7f47d2ac3 P 76cf6e444f59424880d1988ca591c7d5: Deleting consensus metadata
I20260806 04:34:56.450606 22727 catalog_manager.cc:4841] TS 76cf6e444f59424880d1988ca591c7d5 (127.12.15.194:32835): tablet 984c61af04c24bb38b48c0b7f47d2ac3 (table test-table [id=6d7df04a853d447e8c8d5a17ed0f260c]) successfully deleted
I20260806 04:34:56.451923 22725 catalog_manager.cc:4841] TS 5859c5a9b5a64f5cb616be01df292198 (127.12.15.195:46261): tablet 984c61af04c24bb38b48c0b7f47d2ac3 (table test-table [id=6d7df04a853d447e8c8d5a17ed0f260c]) successfully deleted
I20260806 04:34:56.459291 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 23032
W20260806 04:34:56.459311 23561 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:56.458992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36159 (local address 127.0.0.1:34304)
0806 04:34:56.459133 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:56.459190 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:36159: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 04:34:56.465242 22729 catalog_manager.cc:4584] TS abaa6ef902644442aef07e3c40f04693 (127.12.15.193:36159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 984c61af04c24bb38b48c0b7f47d2ac3: Network error: Client connection negotiation failed: client connection to 127.12.15.193:36159: connect: Connection reset by peer (error 104)
I20260806 04:34:56.477125 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 23239
I20260806 04:34:56.488584 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 22274
I20260806 04:34:56.500079 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 22425
2026-08-06T04:34:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5743 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:34:56 dist-test-slave-xt4k krb5kdc[23610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:34:56 dist-test-slave-xt4k krb5kdc[23610](info): set up 2 sockets
Aug 06 04:34:56 dist-test-slave-xt4k krb5kdc[23610](info): commencing operation
krb5kdc: starting...
W20260806 04:34:57.662566 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:34:57 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:34:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:34:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab.
Entry for principal oryx/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab.
I20260806 04:34:57.791019 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33599
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab
--principal=oryx/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:33599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:57.902874 23626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:57.903133 23626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:57.903173 23626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:57.907119 23626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:34:57.907171 23626 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 04:34:57.907186 23626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:57.907203 23626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:57.907218 23626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:34:57.907244 23626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:34:57.912405 23626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:33599
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.254.keytab
--principal=oryx/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:57.913400 23626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:57.914448 23626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:57.922652 23636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:57.922659 23634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:57.922657 23631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:57.923033 23626 server_base.cc:1034] running on GCE node
I20260806 04:34:57.923595 23626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:57.924649 23626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:57.925814 23626 hybrid_clock.cc:648] HybridClock initialized: now 1785990897925789 us; error 36 us; skew 500 ppm
Aug 06 04:34:57 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990897, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:57.928483 23626 init.cc:377] Logged in from keytab as oryx/127.12.15.254@KRBTEST.COM (short username oryx)
I20260806 04:34:57.929318 23626 webserver.cc:460] Webserver started at http://127.12.15.254:43787/ using document root <none> and password file <none>
I20260806 04:34:57.929864 23626 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:57.929930 23626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:57.930119 23626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:57.932145 23626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "419b86a04b654b33aee8ee75f12cb58f"
format_stamp: "Formatted at 2026-08-06 04:34:57 on dist-test-slave-xt4k"
server_key: "ba1130621b4dac3ad0cfb05042fd55af"
server_key_iv: "6e05210788629ffe2c01027f2baf3238"
server_key_version: "encryptionkey@0"
I20260806 04:34:57.932752 23626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "419b86a04b654b33aee8ee75f12cb58f"
format_stamp: "Formatted at 2026-08-06 04:34:57 on dist-test-slave-xt4k"
server_key: "ba1130621b4dac3ad0cfb05042fd55af"
server_key_iv: "6e05210788629ffe2c01027f2baf3238"
server_key_version: "encryptionkey@0"
I20260806 04:34:57.936406 23626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:34:57.938879 23645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:57.940073 23626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 04:34:57.940186 23626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "419b86a04b654b33aee8ee75f12cb58f"
format_stamp: "Formatted at 2026-08-06 04:34:57 on dist-test-slave-xt4k"
server_key: "ba1130621b4dac3ad0cfb05042fd55af"
server_key_iv: "6e05210788629ffe2c01027f2baf3238"
server_key_version: "encryptionkey@0"
I20260806 04:34:57.940279 23626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:57.954931 23626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:57.957887 23626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:57.958069 23626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:57.966125 23626 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:33599
I20260806 04:34:57.966132 23745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:33599 every 8 connection(s)
I20260806 04:34:57.967096 23626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:34:57.967648 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 23626
I20260806 04:34:57.967789 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:34:57.968078 12351 external_mini_cluster.cc:1428] Setting key 903b1a4831678610fae59a7a68d77f85
I20260806 04:34:57.970306 23746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:57.976899 23746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Bootstrap starting.
Aug 06 04:34:57 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:57.979552 23746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:57.980885 23746 log.cc:824] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:57.983201 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:57.969469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.12.15.254:33599)
0806 04:34:57.969895 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:57.969923 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:57.969967 (+ 44us) server_negotiation.cc:407] Connection header received
0806 04:34:57.970802 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:57.970826 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:57.971234 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:57.971600 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:57.972261 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:57.973804 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:57.974662 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:57.975332 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:57.975490 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:57.977881 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:57.977888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:57.977906 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:57.977941 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:57.979956 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:57.980406 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:57.980412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:57.980417 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:57.980569 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:57.980874 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:57.980877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:57.980879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:57.981404 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:57.981566 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:57.981946 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 04:34:57.982255 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":116,"threads_started":1}
I20260806 04:34:57.983292 23746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: No bootstrap required, opened a new log
I20260806 04:34:57.986689 23746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b86a04b654b33aee8ee75f12cb58f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33599 } }
I20260806 04:34:57.986941 23746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:57.987000 23746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419b86a04b654b33aee8ee75f12cb58f, State: Initialized, Role: FOLLOWER
I20260806 04:34:57.987412 23746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b86a04b654b33aee8ee75f12cb58f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33599 } }
I20260806 04:34:57.987579 23746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:34:57.987656 23746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:34:57.987751 23746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:57.988756 23746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b86a04b654b33aee8ee75f12cb58f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33599 } }
I20260806 04:34:57.989071 23746 leader_election.cc:302] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 419b86a04b654b33aee8ee75f12cb58f; no voters:
I20260806 04:34:57.989352 23746 leader_election.cc:288] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:34:57.989593 23756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:57.989840 23756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [term 1 LEADER]: Becoming Leader. State: Replica: 419b86a04b654b33aee8ee75f12cb58f, State: Running, Role: LEADER
I20260806 04:34:57.990136 23756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b86a04b654b33aee8ee75f12cb58f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33599 } }
I20260806 04:34:57.990579 23746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:34:57.991762 23758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 419b86a04b654b33aee8ee75f12cb58f. Latest consensus state: current_term: 1 leader_uuid: "419b86a04b654b33aee8ee75f12cb58f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b86a04b654b33aee8ee75f12cb58f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33599 } } }
I20260806 04:34:57.991889 23758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:57.992209 23757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "419b86a04b654b33aee8ee75f12cb58f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419b86a04b654b33aee8ee75f12cb58f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33599 } } }
I20260806 04:34:57.992306 23757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f [sys.catalog]: This master's current role is: LEADER
I20260806 04:34:57.992771 23768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:34:57.995573 23768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:34:58.001331 23768 catalog_manager.cc:1347] Generated new cluster ID: d9781a0def1f45f4b6efdb62cd8eb982
I20260806 04:34:58.001405 23768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:34:58.011713 23768 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:34:58.012710 23768 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:34:58.016901 23768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Generated new TSK 0
I20260806 04:34:58.017584 23768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab.
Entry for principal oryx/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab.
I20260806 04:34:58.049777 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:33599
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:58.164827 23789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:58.165069 23789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:58.165130 23789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:58.169051 23789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:58.169102 23789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:58.169157 23789 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:58.169205 23789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:34:58.174297 23789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:33599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:58.175307 23789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:58.176313 23789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:58.185884 23802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:58.185969 23798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:58.185884 23799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:58.186110 23789 server_base.cc:1034] running on GCE node
I20260806 04:34:58.186524 23789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:58.187181 23789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:58.188400 23789 hybrid_clock.cc:648] HybridClock initialized: now 1785990898188370 us; error 35 us; skew 500 ppm
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:58.191213 23789 init.cc:377] Logged in from keytab as oryx/127.12.15.193@KRBTEST.COM (short username oryx)
I20260806 04:34:58.192198 23789 webserver.cc:460] Webserver started at http://127.12.15.193:36605/ using document root <none> and password file <none>
I20260806 04:34:58.192829 23789 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:58.192900 23789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:58.193117 23789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:58.195221 23789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "d5344e20885f4f36b32d68f87cef6fe8"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "96a3b22e9e6079372051829d2d335376"
server_key_iv: "3efaf7eb76fd07d09e39d99dd0e4e143"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.195775 23789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "d5344e20885f4f36b32d68f87cef6fe8"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "96a3b22e9e6079372051829d2d335376"
server_key_iv: "3efaf7eb76fd07d09e39d99dd0e4e143"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.199394 23789 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:34:58.201831 23811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:58.203007 23789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:58.203120 23789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "d5344e20885f4f36b32d68f87cef6fe8"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "96a3b22e9e6079372051829d2d335376"
server_key_iv: "3efaf7eb76fd07d09e39d99dd0e4e143"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.203213 23789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:58.223829 23789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:58.226840 23789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:58.227025 23789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:58.228571 23789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:58.228623 23789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:58.228679 23789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:58.228709 23789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:58.239048 23789 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:46609
I20260806 04:34:58.239293 23993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:46609 every 8 connection(s)
I20260806 04:34:58.240159 23789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:34:58.242288 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.229372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:58037 (local address 127.12.15.254:33599)
0806 04:34:58.229525 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.229530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.230383 (+ 853us) server_negotiation.cc:407] Connection header received
0806 04:34:58.231389 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.231393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.231454 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.231514 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.233130 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.233757 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.234948 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.235227 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.235319 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.238176 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.238179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.238181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.238208 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.239821 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.240555 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.240559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.240562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.240672 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.241071 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.241075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.241077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.241317 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.241416 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.241976 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.242118 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:34:58.243090 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.229754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.12.15.193:58037)
0806 04:34:58.230215 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.230265 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.231135 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.231944 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.231953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.232412 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:58.232972 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.232986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.233889 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.233893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.234821 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.234832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.235400 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.235405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.235568 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.236178 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:58.236197 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:58.237983 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:58.239953 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.239959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.239973 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.240422 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.240787 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.240791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.240793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.240958 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.241426 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:58.241433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:58.241729 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.241889 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.242216 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260806 04:34:58.244470 23994 heartbeater.cc:347] Connected to a master server at 127.12.15.254:33599
I20260806 04:34:58.244633 23994 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:58.245189 23994 heartbeater.cc:511] Master 127.12.15.254:33599 requested a full tablet report, sending...
I20260806 04:34:58.246800 23673 ts_manager.cc:194] Registered new tserver with Master: d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609)
I20260806 04:34:58.247687 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 23789
I20260806 04:34:58.247812 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:34:58.248107 12351 external_mini_cluster.cc:1428] Setting key bc899804b44a531d0a7ba8b70719795c
I20260806 04:34:58.248399 23673 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:58037
I20260806 04:34:58.253835 24002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.246427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.12.15.193:48921)
0806 04:34:58.246685 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.246696 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.246778 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.247118 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.247121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.247328 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:58.247448 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.247455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.248678 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.248681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.249504 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.249516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.249946 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.249949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.250048 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.250512 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:58.250526 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:58.250857 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:58.252385 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.252388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.252391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.252601 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.252865 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.252867 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.252869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.252976 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.253392 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:58.253395 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:58.253478 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.253572 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.253678 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
I20260806 04:34:58.253912 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.246488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:48921 (local address 127.12.15.254:33599)
0806 04:34:58.246622 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.246626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.246756 (+ 130us) server_negotiation.cc:407] Connection header received
0806 04:34:58.246862 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.246865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.246923 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.246998 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.247582 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.248380 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.249622 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.249841 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.249933 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.250990 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.250993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.250995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.251016 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.252264 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.252690 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.252694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.252695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.252767 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.253067 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.253070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.253071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.253274 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.253369 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.253641 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.253762 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for oryx/127.12.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab.
Entry for principal oryx/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab.
I20260806 04:34:58.279458 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:33599
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:58.393271 24007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:58.393533 24007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:58.393580 24007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:58.397619 24007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:58.397672 24007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:58.397717 24007 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:58.397770 24007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:34:58.403016 24007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:33599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:58.404058 24007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:58.405113 24007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:34:58.415211 24007 server_base.cc:1034] running on GCE node
W20260806 04:34:58.415160 24013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:58.415167 24014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:58.415138 24018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:58.415784 24007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:58.416461 24007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:58.417624 24007 hybrid_clock.cc:648] HybridClock initialized: now 1785990898417557 us; error 79 us; skew 500 ppm
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:58.420389 24007 init.cc:377] Logged in from keytab as oryx/127.12.15.194@KRBTEST.COM (short username oryx)
I20260806 04:34:58.421267 24007 webserver.cc:460] Webserver started at http://127.12.15.194:42627/ using document root <none> and password file <none>
I20260806 04:34:58.421805 24007 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:58.421872 24007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:58.422089 24007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:58.424117 24007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "04f97d9848aa4dd597e9d6c902cc9ee2"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "809bde36dfab0247900106b2f8249288"
server_key_iv: "44c10ce0f286c52471c912db0af0f821"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.424710 24007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "04f97d9848aa4dd597e9d6c902cc9ee2"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "809bde36dfab0247900106b2f8249288"
server_key_iv: "44c10ce0f286c52471c912db0af0f821"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.428288 24007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:34:58.430688 24032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:58.431952 24007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:58.432078 24007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "04f97d9848aa4dd597e9d6c902cc9ee2"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "809bde36dfab0247900106b2f8249288"
server_key_iv: "44c10ce0f286c52471c912db0af0f821"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.432188 24007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:58.458410 24007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:58.461596 24007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:58.461778 24007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:58.463323 24007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:58.463373 24007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:58.463429 24007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:58.463459 24007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:58.473694 24007 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:37857
I20260806 04:34:58.473752 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:37857 every 8 connection(s)
I20260806 04:34:58.474632 24007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:34:58.476642 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.464124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:37755 (local address 127.12.15.254:33599)
0806 04:34:58.464238 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.464242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.465383 (+ 1141us) server_negotiation.cc:407] Connection header received
0806 04:34:58.466415 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.466418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.466499 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.466562 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.468299 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.468866 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.470031 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.470314 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.470405 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.473123 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.473127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.473129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.473154 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.474589 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.475180 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.475184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.475186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.475287 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.475626 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.475629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.475630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.475811 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.475900 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.476391 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.476520 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 04:34:58.476882 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 24007
I20260806 04:34:58.477017 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:34:58.477340 12351 external_mini_cluster.cc:1428] Setting key aab1f41cf581286dba2b2c98d20eb8a2
I20260806 04:34:58.477587 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.464711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.12.15.194:37755)
0806 04:34:58.465226 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.465267 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.466162 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.466991 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.467001 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.467478 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:58.468155 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.468175 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.468990 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.468994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.469922 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.469933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.470491 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.470496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.470649 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.471168 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:58.471184 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:58.472937 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:58.474702 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.474708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.474722 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.475059 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.475396 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.475399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.475401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.475534 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.475910 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:58.475915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:58.476155 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.476305 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.476591 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":139,"threads_started":1}
I20260806 04:34:58.478816 24205 heartbeater.cc:347] Connected to a master server at 127.12.15.254:33599
I20260806 04:34:58.478956 24205 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:58.479511 24205 heartbeater.cc:511] Master 127.12.15.254:33599 requested a full tablet report, sending...
I20260806 04:34:58.480580 23674 ts_manager.cc:194] Registered new tserver with Master: 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857)
I20260806 04:34:58.481208 23674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:37755
WARNING: no policy specified for oryx/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:34:58.489391 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.481623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.12.15.194:59189)
0806 04:34:58.481925 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.481947 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.482036 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.482917 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.482920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.483096 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:58.483183 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.483188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.484081 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.484084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.484673 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.484683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.485203 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.485206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.485273 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.485703 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:58.485715 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:58.486023 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:58.487871 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.487874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.487875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.488076 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.488373 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.488376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.488378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.488456 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.488933 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:58.488936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:58.489021 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.489105 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.489210 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260806 04:34:58.489518 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.481654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:59189 (local address 127.12.15.254:33599)
0806 04:34:58.481758 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.481762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.482011 (+ 249us) server_negotiation.cc:407] Connection header received
0806 04:34:58.482778 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.482782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.482827 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.482888 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.483286 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.483958 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.484790 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.485099 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.485200 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.486159 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.486163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.486166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.486191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.487753 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.488174 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.488178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.488180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.488264 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.488539 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.488542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.488543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.488829 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.488933 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.489176 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.489350 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab.
Entry for principal oryx/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab.
I20260806 04:34:58.508167 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:33599
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:34:58.621671 24221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:34:58.621917 24221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:34:58.621961 24221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:34:58.626011 24221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:34:58.626063 24221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:34:58.626103 24221 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 04:34:58.626154 24221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:34:58.631280 24221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.254:33599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:34:58.632266 24221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:34:58.633277 24221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:34:58.643179 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:58.643190 24229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:34:58.643190 24231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:34:58.643491 24221 server_base.cc:1034] running on GCE node
I20260806 04:34:58.644017 24221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:34:58.644701 24221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:34:58.645862 24221 hybrid_clock.cc:648] HybridClock initialized: now 1785990898645833 us; error 40 us; skew 500 ppm
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:34:58.648592 24221 init.cc:377] Logged in from keytab as oryx/127.12.15.195@KRBTEST.COM (short username oryx)
I20260806 04:34:58.649524 24221 webserver.cc:460] Webserver started at http://127.12.15.195:35893/ using document root <none> and password file <none>
I20260806 04:34:58.650101 24221 fs_manager.cc:359] Metadata directory not provided
I20260806 04:34:58.650170 24221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:34:58.650377 24221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:34:58.652491 24221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "da55e449b8d94f33bff4a3a2db57fd6e"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "3ba4502e249576f9d4c790046be90da2"
server_key_iv: "12c75dbf715c59993697abab1e6f4162"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.653061 24221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "da55e449b8d94f33bff4a3a2db57fd6e"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "3ba4502e249576f9d4c790046be90da2"
server_key_iv: "12c75dbf715c59993697abab1e6f4162"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.656885 24221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 04:34:58.659376 24245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:58.660673 24221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:34:58.660789 24221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "da55e449b8d94f33bff4a3a2db57fd6e"
format_stamp: "Formatted at 2026-08-06 04:34:58 on dist-test-slave-xt4k"
server_key: "3ba4502e249576f9d4c790046be90da2"
server_key_iv: "12c75dbf715c59993697abab1e6f4162"
server_key_version: "encryptionkey@0"
I20260806 04:34:58.660885 24221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:34:58.681615 24221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:34:58.684705 24221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:34:58.684892 24221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:34:58.686432 24221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:34:58.686494 24221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:34:58.686554 24221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:34:58.686581 24221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:58.697255 24221 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:44527
I20260806 04:34:58.698073 24428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:44527 every 8 connection(s)
I20260806 04:34:58.698176 24221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:34:58.699879 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.687252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33837 (local address 127.12.15.254:33599)
0806 04:34:58.687375 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.687378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.688435 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 04:34:58.689462 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.689465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.689525 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.689581 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.691062 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.691646 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.693075 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.693296 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.693397 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.696372 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.696377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.696380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.696408 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.697816 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.698338 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.698342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.698344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.698423 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.698791 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.698794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.698795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.699000 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.699101 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.699588 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.699715 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:34:58.700644 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.687756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.12.15.195:33837)
0806 04:34:58.688261 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.688303 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.689228 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.689991 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.689999 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.690339 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:58.690919 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.690936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.691773 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.691777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.692958 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.692969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.693461 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.693466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.693689 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.694356 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:58.694376 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:58.696178 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:58.697930 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.697936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.697952 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.698246 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.698506 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.698509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.698511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.698678 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.699103 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:58.699110 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:58.699359 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.699523 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.699838 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":139,"threads_started":1}
I20260806 04:34:58.701769 24429 heartbeater.cc:347] Connected to a master server at 127.12.15.254:33599
I20260806 04:34:58.701903 24429 heartbeater.cc:464] Registering TS with master...
I20260806 04:34:58.702469 24429 heartbeater.cc:511] Master 127.12.15.254:33599 requested a full tablet report, sending...
I20260806 04:34:58.703553 23670 ts_manager.cc:194] Registered new tserver with Master: da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527)
I20260806 04:34:58.704267 23670 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:33837
I20260806 04:34:58.705076 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 24221
I20260806 04:34:58.705216 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:34:58.705580 12351 external_mini_cluster.cc:1428] Setting key 118e7a040ebf5cd3feedba2e41c32788
I20260806 04:34:58.707509 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:34:58.710692 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.703914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.12.15.195:45883)
0806 04:34:58.704178 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.704189 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.704278 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.704505 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.704508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.704741 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:58.704884 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.704892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.705718 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.705721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.706413 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.706424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.706897 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.706901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.706990 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.707409 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:58.707422 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:58.707817 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:58.709261 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.709265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.709267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.709492 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.709777 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:58.709780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:58.709782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:58.709867 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:58.710204 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:58.710208 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:58.710294 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.710399 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.710529 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260806 04:34:58.710747 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.703982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:45883 (local address 127.12.15.254:33599)
0806 04:34:58.704090 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.704094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.704250 (+ 156us) server_negotiation.cc:407] Connection header received
0806 04:34:58.704368 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.704372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.704416 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.704481 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.704989 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.705564 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.706510 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.706796 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.706882 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.707926 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.707929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.707931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.707953 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.709159 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.709602 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.709605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.709607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.709682 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.709949 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.709953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.709954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.710116 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.710202 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.710491 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.710606 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:34:58.732457 23751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:58.726633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.12.15.254:33599)
0806 04:34:58.726749 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.726752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.726767 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:34:58.726809 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.726812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.726853 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.726936 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.727532 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.728072 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.728984 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.729212 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.729293 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.732299 (+ 3006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45784: BlockingRecv error: recv got EOF from 127.0.0.1:45784 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 04:34:58 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990898, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:58.740790 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.733731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.12.15.254:33599)
0806 04:34:58.733834 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.733838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.733955 (+ 117us) server_negotiation.cc:407] Connection header received
0806 04:34:58.734067 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.734070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.734114 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.734193 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:58.734693 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.735204 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.735951 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.736169 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.736246 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.738338 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:58.738342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:58.738343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:58.738367 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:58.739536 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.739882 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.739884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.739885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.739951 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:58.740130 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:58.740132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:58.740133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:58.740286 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:58.740428 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.740577 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.740674 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 04:34:58.743631 23670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:58.747094 23670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:58.761052 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.755538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:37857 (local address 127.0.0.1:47098)
0806 04:34:58.756686 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.756698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.756786 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.757258 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.757260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.757278 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.757361 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.757367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.758361 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.758365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.759541 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.759551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.760521 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.760525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.760631 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.760676 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.760786 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.760905 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":151,"threads_started":1}
I20260806 04:34:58.761240 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.755460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33366 (local address 127.12.15.195:44527)
0806 04:34:58.755920 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.755932 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.756396 (+ 464us) server_negotiation.cc:407] Connection header received
0806 04:34:58.756682 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.756689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.756916 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.757066 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.757354 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.758422 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.759891 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.760694 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.760787 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.760944 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.760998 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.761086 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":130,"threads_started":1}
I20260806 04:34:58.761406 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.755272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:44527 (local address 127.0.0.1:33366)
0806 04:34:58.756234 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.756283 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.756515 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.757058 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.757064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.757086 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.757218 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.757226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.758577 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.758580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.759766 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.759776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.760801 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.760805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.760891 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.760914 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.760973 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.761039 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":80,"threads_started":1}
I20260806 04:34:58.761502 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.755745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.12.15.194:37857)
0806 04:34:58.756168 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.756183 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.756786 (+ 603us) server_negotiation.cc:407] Connection header received
0806 04:34:58.756887 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.756892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.757079 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.757268 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.757450 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.758252 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.759660 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.760408 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.760517 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.760691 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.761268 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.761359 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":109,"threads_started":1}
I20260806 04:34:58.761941 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.755959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:46609 (local address 127.0.0.1:58036)
0806 04:34:58.756437 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.756449 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.756536 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.757538 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.757541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.757559 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.757653 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.757659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.758963 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.758966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.760004 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.760011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.761051 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.761055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.761653 (+ 598us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.761679 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.761758 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.761824 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":96,"threads_started":1}
I20260806 04:34:58.762537 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.756389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.12.15.193:46609)
0806 04:34:58.757000 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.757006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.757025 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:34:58.757102 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.757107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.757277 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.757404 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.757777 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.758841 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.760104 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.760776 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.760862 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.760991 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.762291 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.762392 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":89,"threads_started":1}
I20260806 04:34:58.764454 24306 tablet_service.cc:1467] Processing CreateTablet for tablet dc07409b7b7c4ad58dd8dac5766dc425 (DEFAULT_TABLE table=test-table [id=96087303125443a5ac223d0a08f62f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:58.764470 24106 tablet_service.cc:1467] Processing CreateTablet for tablet dc07409b7b7c4ad58dd8dac5766dc425 (DEFAULT_TABLE table=test-table [id=96087303125443a5ac223d0a08f62f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:58.764778 23891 tablet_service.cc:1467] Processing CreateTablet for tablet dc07409b7b7c4ad58dd8dac5766dc425 (DEFAULT_TABLE table=test-table [id=96087303125443a5ac223d0a08f62f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:58.765590 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc07409b7b7c4ad58dd8dac5766dc425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:58.765588 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc07409b7b7c4ad58dd8dac5766dc425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:58.765782 23891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc07409b7b7c4ad58dd8dac5766dc425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:58.771729 24470 tablet_bootstrap.cc:492] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Bootstrap starting.
I20260806 04:34:58.773744 24472 tablet_bootstrap.cc:492] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Bootstrap starting.
I20260806 04:34:58.773998 24470 tablet_bootstrap.cc:654] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:58.774693 24470 log.cc:824] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:58.775388 24472 tablet_bootstrap.cc:654] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:58.776019 24472 log.cc:824] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:58.776355 24470 tablet_bootstrap.cc:492] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: No bootstrap required, opened a new log
I20260806 04:34:58.776539 24470 ts_tablet_manager.cc:1397] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 04:34:58.776639 24473 tablet_bootstrap.cc:492] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Bootstrap starting.
I20260806 04:34:58.777894 24472 tablet_bootstrap.cc:492] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: No bootstrap required, opened a new log
I20260806 04:34:58.778083 24472 ts_tablet_manager.cc:1397] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:34:58.778246 24473 tablet_bootstrap.cc:654] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:58.778930 24473 log.cc:824] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Log is configured to *not* fsync() on all Append() calls
I20260806 04:34:58.780586 24473 tablet_bootstrap.cc:492] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: No bootstrap required, opened a new log
I20260806 04:34:58.780751 24473 ts_tablet_manager.cc:1397] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 04:34:58.781011 24470 raft_consensus.cc:348] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.781288 24470 raft_consensus.cc:374] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:58.781347 24470 raft_consensus.cc:729] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5344e20885f4f36b32d68f87cef6fe8, State: Initialized, Role: FOLLOWER
I20260806 04:34:58.781800 24470 consensus_queue.cc:260] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.781996 24472 raft_consensus.cc:348] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.782238 24472 raft_consensus.cc:374] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:58.782294 24472 raft_consensus.cc:729] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Initialized, Role: FOLLOWER
I20260806 04:34:58.782793 24470 ts_tablet_manager.cc:1428] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:34:58.782717 24472 consensus_queue.cc:260] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.783159 23994 heartbeater.cc:503] Master 127.12.15.254:33599 was elected leader, sending a full tablet report...
I20260806 04:34:58.783648 24472 ts_tablet_manager.cc:1428] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:34:58.783979 24429 heartbeater.cc:503] Master 127.12.15.254:33599 was elected leader, sending a full tablet report...
I20260806 04:34:58.783855 24473 raft_consensus.cc:348] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.784116 24473 raft_consensus.cc:374] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:58.784173 24473 raft_consensus.cc:729] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Initialized, Role: FOLLOWER
I20260806 04:34:58.784619 24473 consensus_queue.cc:260] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.785424 24205 heartbeater.cc:503] Master 127.12.15.254:33599 was elected leader, sending a full tablet report...
I20260806 04:34:58.785526 24473 ts_tablet_manager.cc:1428] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 04:34:58.854055 24480 raft_consensus.cc:482] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:58.854261 24480 raft_consensus.cc:504] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.855422 24480 leader_election.cc:288] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857), d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609)
I20260806 04:34:58.858971 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.855661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:46609 (local address 127.12.15.195:49209)
0806 04:34:58.855820 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.855835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.855929 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.856233 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.856236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.856255 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.856397 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.856404 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.857125 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.857129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.857904 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.857913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.858563 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.858567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.858678 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.858698 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.858763 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.858834 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":3}
I20260806 04:34:58.858971 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.855899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:49209 (local address 127.12.15.193:46609)
0806 04:34:58.856041 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.856045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.856056 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:34:58.856101 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.856103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.856141 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.856206 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.856506 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.857013 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.858037 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.858449 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.858562 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.858620 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.858795 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.858871 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:34:58.859468 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.855661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:37857 (local address 127.12.15.195:33115)
0806 04:34:58.856164 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.856175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.856275 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.856513 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.856515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.856531 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.856985 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.856992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.857702 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.857706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.858462 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.858471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.859105 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.859109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.859204 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.859225 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.859291 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.859352 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":127,"thread_start_us":154,"threads_started":1}
I20260806 04:34:58.859505 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.855796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33115 (local address 127.12.15.194:37857)
0806 04:34:58.855917 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.855921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.856259 (+ 338us) server_negotiation.cc:407] Connection header received
0806 04:34:58.856382 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.856385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.856429 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.856491 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.857107 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.857595 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.858588 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.858973 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.859077 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.859125 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.859319 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.859398 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:34:58.859635 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc07409b7b7c4ad58dd8dac5766dc425" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8" is_pre_election: true
I20260806 04:34:58.859982 23921 raft_consensus.cc:2454] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 0.
I20260806 04:34:58.860109 24137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc07409b7b7c4ad58dd8dac5766dc425" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" is_pre_election: true
I20260806 04:34:58.860404 24137 raft_consensus.cc:2454] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 0.
I20260806 04:34:58.860476 24248 leader_election.cc:302] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5344e20885f4f36b32d68f87cef6fe8, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:58.860744 24480 raft_consensus.cc:2781] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:58.860823 24480 raft_consensus.cc:482] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:58.860857 24480 raft_consensus.cc:3037] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:58.861864 24480 raft_consensus.cc:504] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.862182 24480 leader_election.cc:288] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 election: Requested vote from peers 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857), d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609)
I20260806 04:34:58.862512 24137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc07409b7b7c4ad58dd8dac5766dc425" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2"
I20260806 04:34:58.862581 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc07409b7b7c4ad58dd8dac5766dc425" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8"
I20260806 04:34:58.862623 24137 raft_consensus.cc:3037] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:58.862697 23921 raft_consensus.cc:3037] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:58.863644 24137 raft_consensus.cc:2454] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 1.
I20260806 04:34:58.863802 23921 raft_consensus.cc:2454] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 1.
I20260806 04:34:58.863945 24247 leader_election.cc:302] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:58.864125 24480 raft_consensus.cc:2781] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:58.864323 24480 raft_consensus.cc:686] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 LEADER]: Becoming Leader. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Running, Role: LEADER
I20260806 04:34:58.864688 24480 consensus_queue.cc:237] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } }
I20260806 04:34:58.867861 23670 catalog_manager.cc:5495] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e reported cstate change: term changed from 0 to 1, leader changed from <none> to da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195). New cstate: current_term: 1 leader_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:58.885142 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.881507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.12.15.195:44527)
0806 04:34:58.881660 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.881664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.881677 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:58.881752 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.881754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.881807 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.881895 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:58.882186 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.882725 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.883631 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.883864 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.883947 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.884228 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:58.884764 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:58.884855 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.884958 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.885026 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 04:34:58.891212 23921 raft_consensus.cc:1261] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Refusing update from remote peer da55e449b8d94f33bff4a3a2db57fd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:58.891556 24137 raft_consensus.cc:1261] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Refusing update from remote peer da55e449b8d94f33bff4a3a2db57fd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:58.892021 24486 consensus_queue.cc:1035] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:58.892287 24480 consensus_queue.cc:1035] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:58.900288 24491 mvcc.cc:205] Tried to move back new op lower bound from 7315418721850880000 to 7315418721752281088. Current Snapshot: MvccSnapshot[applied={T|T < 7315418721850880000}]
I20260806 04:34:58.909909 23674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:58.910072 23674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:58.912495 23674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Sending DeleteTablet for 3 replicas of tablet dc07409b7b7c4ad58dd8dac5766dc425
I20260806 04:34:58.913341 23891 tablet_service.cc:1514] Processing DeleteTablet for tablet dc07409b7b7c4ad58dd8dac5766dc425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:58 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:58036
I20260806 04:34:58.913335 24106 tablet_service.cc:1514] Processing DeleteTablet for tablet dc07409b7b7c4ad58dd8dac5766dc425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:58 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47098
I20260806 04:34:58.914077 24507 tablet_replica.cc:330] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: stopping tablet replica
I20260806 04:34:58.914036 24306 tablet_service.cc:1514] Processing DeleteTablet for tablet dc07409b7b7c4ad58dd8dac5766dc425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:58 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:33366
I20260806 04:34:58.914149 23674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 04:34:58.914527 24507 raft_consensus.cc:2229] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:58.914572 24509 tablet_replica.cc:330] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: stopping tablet replica
W20260806 04:34:58.914572 23674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:58.914825 24509 raft_consensus.cc:2229] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:58.915021 24507 raft_consensus.cc:2258] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:58.915225 24509 raft_consensus.cc:2258] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:58.915508 24503 tablet_replica.cc:330] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: stopping tablet replica
I20260806 04:34:58.915832 24503 raft_consensus.cc:2229] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:58.917073 24507 ts_tablet_manager.cc:1905] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:58.918215 24503 raft_consensus.cc:2258] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:58.919934 24507 ts_tablet_manager.cc:1918] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:58.919955 23891 tablet_service.cc:1467] Processing CreateTablet for tablet af9f78a8774846b7a50d1cff1b7455f5 (DEFAULT_TABLE table=test-table [id=c28e4bf8138743ee8680c9734f11a116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:58.920032 24507 log.cc:1196] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/dc07409b7b7c4ad58dd8dac5766dc425
I20260806 04:34:58.920203 23891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9f78a8774846b7a50d1cff1b7455f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:58.920377 24507 ts_tablet_manager.cc:1939] T dc07409b7b7c4ad58dd8dac5766dc425 P d5344e20885f4f36b32d68f87cef6fe8: Deleting consensus metadata
I20260806 04:34:58.919955 24106 tablet_service.cc:1467] Processing CreateTablet for tablet af9f78a8774846b7a50d1cff1b7455f5 (DEFAULT_TABLE table=test-table [id=c28e4bf8138743ee8680c9734f11a116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:58.920796 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9f78a8774846b7a50d1cff1b7455f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:58.922278 24470 tablet_bootstrap.cc:492] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Bootstrap starting.
I20260806 04:34:58.922479 24306 tablet_service.cc:1467] Processing CreateTablet for tablet af9f78a8774846b7a50d1cff1b7455f5 (DEFAULT_TABLE table=test-table [id=c28e4bf8138743ee8680c9734f11a116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:58.922748 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9f78a8774846b7a50d1cff1b7455f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:58.923264 24470 tablet_bootstrap.cc:654] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:58.923840 24473 tablet_bootstrap.cc:492] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Bootstrap starting.
I20260806 04:34:58.924832 24473 tablet_bootstrap.cc:654] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:58.924921 24472 tablet_bootstrap.cc:492] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Bootstrap starting.
I20260806 04:34:58.925607 23647 catalog_manager.cc:4841] TS d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609): tablet dc07409b7b7c4ad58dd8dac5766dc425 (table test-table [id=96087303125443a5ac223d0a08f62f57]) successfully deleted
I20260806 04:34:58.925719 24470 tablet_bootstrap.cc:492] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: No bootstrap required, opened a new log
I20260806 04:34:58.925791 24470 ts_tablet_manager.cc:1397] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 04:34:58.925868 24472 tablet_bootstrap.cc:654] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:58.926218 24470 raft_consensus.cc:348] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.926331 24470 raft_consensus.cc:374] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:58.926362 24470 raft_consensus.cc:729] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5344e20885f4f36b32d68f87cef6fe8, State: Initialized, Role: FOLLOWER
I20260806 04:34:58.926474 24470 consensus_queue.cc:260] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.926721 24470 ts_tablet_manager.cc:1428] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:58.926713 24509 ts_tablet_manager.cc:1905] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:58.928251 24503 ts_tablet_manager.cc:1905] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:58.930891 24503 ts_tablet_manager.cc:1918] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:58.930997 24503 log.cc:1196] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/dc07409b7b7c4ad58dd8dac5766dc425
I20260806 04:34:58.931259 24509 ts_tablet_manager.cc:1918] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:58.931324 24503 ts_tablet_manager.cc:1939] T dc07409b7b7c4ad58dd8dac5766dc425 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting consensus metadata
I20260806 04:34:58.931360 24509 log.cc:1196] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/dc07409b7b7c4ad58dd8dac5766dc425
I20260806 04:34:58.932065 24509 ts_tablet_manager.cc:1939] T dc07409b7b7c4ad58dd8dac5766dc425 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting consensus metadata
I20260806 04:34:58.932703 23646 catalog_manager.cc:4841] TS 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857): tablet dc07409b7b7c4ad58dd8dac5766dc425 (table test-table [id=96087303125443a5ac223d0a08f62f57]) successfully deleted
I20260806 04:34:58.933250 23647 catalog_manager.cc:4841] TS da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527): tablet dc07409b7b7c4ad58dd8dac5766dc425 (table test-table [id=96087303125443a5ac223d0a08f62f57]) successfully deleted
I20260806 04:34:58.933591 24473 tablet_bootstrap.cc:492] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: No bootstrap required, opened a new log
I20260806 04:34:58.933689 24473 ts_tablet_manager.cc:1397] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 04:34:58.934118 24472 tablet_bootstrap.cc:492] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: No bootstrap required, opened a new log
I20260806 04:34:58.934106 24473 raft_consensus.cc:348] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.934194 24473 raft_consensus.cc:374] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:58.934195 24472 ts_tablet_manager.cc:1397] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260806 04:34:58.934216 24473 raft_consensus.cc:729] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Initialized, Role: FOLLOWER
I20260806 04:34:58.934322 24473 consensus_queue.cc:260] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.934600 24473 ts_tablet_manager.cc:1428] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:58.934621 24472 raft_consensus.cc:348] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.934737 24472 raft_consensus.cc:374] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:58.934779 24472 raft_consensus.cc:729] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Initialized, Role: FOLLOWER
I20260806 04:34:58.934904 24472 consensus_queue.cc:260] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.935166 24472 ts_tablet_manager.cc:1428] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260806 04:34:58.949112 24431 tablet.cc:2369] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:58.974956 24482 raft_consensus.cc:482] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:58.975112 24482 raft_consensus.cc:504] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
W20260806 04:34:58.975893 24207 tablet.cc:2369] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:34:58.976424 24482 leader_election.cc:288] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527)
I20260806 04:34:58.981966 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.977707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:41885 (local address 127.12.15.195:44527)
0806 04:34:58.977845 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.977849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.977903 (+ 54us) server_negotiation.cc:407] Connection header received
0806 04:34:58.978085 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.978088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.978135 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.978179 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.978585 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.979248 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.980673 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.981254 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.981347 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.981403 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.981748 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.981830 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:34:58.982048 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.976679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:46609 (local address 127.12.15.194:55321)
0806 04:34:58.977321 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.977338 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.977425 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.977768 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.977771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.977789 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.977907 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.977914 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.978967 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.978970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.980197 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.980209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.981682 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.981686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.981783 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.981801 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.981858 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.981928 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558}
I20260806 04:34:58.982333 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.976933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:44527 (local address 127.12.15.194:41885)
0806 04:34:58.977827 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:58.977839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:58.977973 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:58.978305 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:58.978307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:58.978323 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:34:58.978453 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.978467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.979387 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.979390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.980535 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:58.980546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.981512 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.981515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:58.981611 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.981634 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:34:58.982130 (+ 496us) client_negotiation.cc:240] Negotiation successful
0806 04:34:58.982204 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":71,"threads_started":1}
I20260806 04:34:58.982728 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:58.977289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:55321 (local address 127.12.15.193:46609)
0806 04:34:58.977403 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:58.977407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:58.977421 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:58.977535 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:58.977539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:58.977587 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:58.977650 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:34:58.978055 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.978809 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.980390 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:58.981477 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:58.981582 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:58.981632 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:58.982522 (+ 890us) server_negotiation.cc:299] Negotiation successful
0806 04:34:58.982611 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:34:58.983067 24364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f78a8774846b7a50d1cff1b7455f5" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" is_pre_election: true
I20260806 04:34:58.983335 24364 raft_consensus.cc:2454] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 0.
I20260806 04:34:58.983659 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f78a8774846b7a50d1cff1b7455f5" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8" is_pre_election: true
I20260806 04:34:58.983798 23921 raft_consensus.cc:2454] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 0.
I20260806 04:34:58.983955 24035 leader_election.cc:302] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:58.984232 24482 raft_consensus.cc:2781] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:58.984292 24482 raft_consensus.cc:482] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:58.984326 24482 raft_consensus.cc:3037] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:58.985420 24482 raft_consensus.cc:504] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.985761 24482 leader_election.cc:288] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 election: Requested vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527)
I20260806 04:34:58.986101 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f78a8774846b7a50d1cff1b7455f5" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8"
I20260806 04:34:58.986225 23921 raft_consensus.cc:3037] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:58.986265 24364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f78a8774846b7a50d1cff1b7455f5" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da55e449b8d94f33bff4a3a2db57fd6e"
I20260806 04:34:58.986382 24364 raft_consensus.cc:3037] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:58.987601 23921 raft_consensus.cc:2454] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 1.
I20260806 04:34:58.987632 24364 raft_consensus.cc:2454] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 1.
I20260806 04:34:58.987922 24035 leader_election.cc:302] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, d5344e20885f4f36b32d68f87cef6fe8; no voters:
I20260806 04:34:58.988103 24482 raft_consensus.cc:2781] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:58.988351 24482 raft_consensus.cc:686] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 LEADER]: Becoming Leader. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Running, Role: LEADER
I20260806 04:34:58.988631 24482 consensus_queue.cc:237] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:58.991381 23670 catalog_manager.cc:5495] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } health_report { overall_health: UNKNOWN } } }
W20260806 04:34:58.991554 23995 tablet.cc:2369] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 04:34:59 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990897, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.254@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:59.027141 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.018381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.12.15.254:33599)
0806 04:34:59.018548 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.018553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.018717 (+ 164us) server_negotiation.cc:407] Connection header received
0806 04:34:59.018868 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.018870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.018907 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.018985 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:59.019691 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.020207 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.021046 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.021253 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.021340 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.024052 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:59.024055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:59.024057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:59.024079 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:59.025474 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:59.025919 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:59.025923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:59.025925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:59.025995 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:59.026231 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:59.026234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:59.026236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:59.026411 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:59.026503 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.026816 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.026983 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":82}
I20260806 04:34:59.027158 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.018317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33599 (local address 127.0.0.1:45792)
0806 04:34:59.018631 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:59.018643 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:59.018770 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:59.019022 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:59.019025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:59.019462 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:34:59.019572 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.019579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.020307 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.020310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.020925 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.020936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.021358 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.021361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.021445 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.022138 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:34:59.022153 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:34:59.023907 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:34:59.025578 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:59.025587 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:59.025594 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:59.025821 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:59.026072 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:34:59.026075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:34:59.026077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:34:59.026157 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:34:59.026515 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:34:59.026522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:34:59.026654 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 04:34:59.026755 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 04:34:59.026939 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":130,"threads_started":1}
I20260806 04:34:59.031591 23670 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45792:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 04:34:59.032414 23670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:59.037818 23891 tablet_service.cc:1467] Processing CreateTablet for tablet 62942f141f614e5785ebd995acc84daf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1f52504c1a44d2ebb1c524a34ec4d99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:59.038023 24306 tablet_service.cc:1467] Processing CreateTablet for tablet 62942f141f614e5785ebd995acc84daf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1f52504c1a44d2ebb1c524a34ec4d99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:59.038305 23891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62942f141f614e5785ebd995acc84daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.038411 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62942f141f614e5785ebd995acc84daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.040922 24472 tablet_bootstrap.cc:492] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: Bootstrap starting.
I20260806 04:34:59.041267 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 62942f141f614e5785ebd995acc84daf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1f52504c1a44d2ebb1c524a34ec4d99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 04:34:59.041615 24470 tablet_bootstrap.cc:492] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: Bootstrap starting.
I20260806 04:34:59.041662 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62942f141f614e5785ebd995acc84daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.042289 24472 tablet_bootstrap.cc:654] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.042650 24470 tablet_bootstrap.cc:654] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.044031 24470 tablet_bootstrap.cc:492] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: No bootstrap required, opened a new log
I20260806 04:34:59.044138 24470 ts_tablet_manager.cc:1397] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:59.044646 24470 raft_consensus.cc:348] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.044776 24470 raft_consensus.cc:374] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.044812 24470 raft_consensus.cc:729] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5344e20885f4f36b32d68f87cef6fe8, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.044833 24473 tablet_bootstrap.cc:492] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: Bootstrap starting.
I20260806 04:34:59.044939 24470 consensus_queue.cc:260] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.045260 24479 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.045401 24479 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.045755 24473 tablet_bootstrap.cc:654] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.045809 24472 tablet_bootstrap.cc:492] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: No bootstrap required, opened a new log
I20260806 04:34:59.045883 24472 ts_tablet_manager.cc:1397] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 04:34:59.046304 24472 raft_consensus.cc:348] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.046414 24472 raft_consensus.cc:374] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.046440 24472 raft_consensus.cc:729] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.046566 24472 consensus_queue.cc:260] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.046818 24472 ts_tablet_manager.cc:1428] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:59.047005 24498 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.047137 24498 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.047215 24470 ts_tablet_manager.cc:1428] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 04:34:59.048220 24473 tablet_bootstrap.cc:492] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: No bootstrap required, opened a new log
I20260806 04:34:59.048307 24473 ts_tablet_manager.cc:1397] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 04:34:59.048774 24473 raft_consensus.cc:348] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.048882 24473 raft_consensus.cc:374] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.048918 24473 raft_consensus.cc:729] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.049034 24473 consensus_queue.cc:260] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.049288 24482 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.049427 24482 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.049302 24473 ts_tablet_manager.cc:1428] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:59.211905 24482 raft_consensus.cc:482] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:59.212045 24482 raft_consensus.cc:504] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.212507 24482 leader_election.cc:288] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527)
I20260806 04:34:59.213047 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62942f141f614e5785ebd995acc84daf" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8" is_pre_election: true
I20260806 04:34:59.213126 24364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62942f141f614e5785ebd995acc84daf" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" is_pre_election: true
I20260806 04:34:59.213210 23921 raft_consensus.cc:2454] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 0.
I20260806 04:34:59.213284 24364 raft_consensus.cc:2454] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 0.
I20260806 04:34:59.213550 24035 leader_election.cc:302] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, d5344e20885f4f36b32d68f87cef6fe8; no voters:
I20260806 04:34:59.213740 24482 raft_consensus.cc:2781] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:59.213798 24482 raft_consensus.cc:482] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:59.213827 24482 raft_consensus.cc:3037] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.214794 24482 raft_consensus.cc:504] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.215095 24482 leader_election.cc:288] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 election: Requested vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527)
I20260806 04:34:59.215409 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62942f141f614e5785ebd995acc84daf" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8"
I20260806 04:34:59.215481 24364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62942f141f614e5785ebd995acc84daf" candidate_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da55e449b8d94f33bff4a3a2db57fd6e"
I20260806 04:34:59.215562 23921 raft_consensus.cc:3037] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.215591 24364 raft_consensus.cc:3037] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.216437 23921 raft_consensus.cc:2454] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 1.
I20260806 04:34:59.216583 24364 raft_consensus.cc:2454] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f97d9848aa4dd597e9d6c902cc9ee2 in term 1.
I20260806 04:34:59.216758 24035 leader_election.cc:302] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, d5344e20885f4f36b32d68f87cef6fe8; no voters:
I20260806 04:34:59.216926 24482 raft_consensus.cc:2781] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:59.217016 24482 raft_consensus.cc:686] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 LEADER]: Becoming Leader. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Running, Role: LEADER
I20260806 04:34:59.217154 24482 consensus_queue.cc:237] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.217738 24519 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: TxnStatusTablet state changed. Reason: New leader 04f97d9848aa4dd597e9d6c902cc9ee2. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.217856 24519 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:59.218132 24533 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 04:34:59.218286 24533 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 04:34:59.218833 23670 catalog_manager.cc:5495] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } health_report { overall_health: UNKNOWN } } }
I20260806 04:34:59.246533 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.241779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47112 (local address 127.12.15.194:37857)
0806 04:34:59.241916 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.241930 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.241987 (+ 57us) server_negotiation.cc:407] Connection header received
0806 04:34:59.242245 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.242249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.242318 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.242425 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:59.242734 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.243511 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.244813 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.245159 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.245242 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.245618 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.246131 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.246245 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.246342 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.246399 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:34:59.246568 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.241680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:37857 (local address 127.0.0.1:47112)
0806 04:34:59.241927 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:59.241942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:59.242063 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:59.242449 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:59.242452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:59.242461 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:59.242574 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.242580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.243670 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.243673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.244644 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.244657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.245324 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.245327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.245412 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.245464 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.246235 (+ 771us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.246254 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 04:34:59.246333 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 04:34:59.246424 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 04:34:59.249614 23921 raft_consensus.cc:1261] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Refusing update from remote peer 04f97d9848aa4dd597e9d6c902cc9ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.249835 24364 raft_consensus.cc:1261] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Refusing update from remote peer 04f97d9848aa4dd597e9d6c902cc9ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.250088 24482 consensus_queue.cc:1035] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.250581 24519 consensus_queue.cc:1035] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.251459 24479 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: TxnStatusTablet state changed. Reason: New leader 04f97d9848aa4dd597e9d6c902cc9ee2. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.251572 24479 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.252241 24498 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: TxnStatusTablet state changed. Reason: New leader 04f97d9848aa4dd597e9d6c902cc9ee2. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.252372 24498 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.254004 24482 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.254119 24482 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:59.255743 24489 mvcc.cc:205] Tried to move back new op lower bound from 7315418723321585664 to 7315418723194658816. Current Snapshot: MvccSnapshot[applied={T|T < 7315418723321585664}]
I20260806 04:34:59.255865 24498 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.255977 24498 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P da55e449b8d94f33bff4a3a2db57fd6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.258383 24519 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.258500 24519 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P 04f97d9848aa4dd597e9d6c902cc9ee2: This TxnStatusTablet replica's current role is: LEADER
I20260806 04:34:59.259119 24479 tablet_replica.cc:439] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } } }
I20260806 04:34:59.259234 24479 tablet_replica.cc:442] T 62942f141f614e5785ebd995acc84daf P d5344e20885f4f36b32d68f87cef6fe8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 04:34:59.269107 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.264610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.12.15.194:37857)
0806 04:34:59.264749 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.264753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.264767 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:34:59.264812 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.264815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.264863 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.264948 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:59.265322 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.266086 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.267395 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.267809 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.267914 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.268309 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.268424 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.268495 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.268939 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.268991 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 04:34:59.280889 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.277885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33711 (local address 127.12.15.194:37857)
0806 04:34:59.278012 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.278016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.278029 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:34:59.278149 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.278152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.278195 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.278255 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:59.278487 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.279011 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.279868 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.280090 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.280177 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.280435 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.280507 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.280564 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.280649 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.280712 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":57}
I20260806 04:34:59.280900 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.277605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:37857 (local address 127.12.15.194:33711)
0806 04:34:59.277923 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:59.277941 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:59.278046 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:59.278268 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:59.278271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:59.278290 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:59.278397 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.278404 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.279114 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.279117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.279769 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.279779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.280180 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.280183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.280257 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.280298 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.280574 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.280579 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 04:34:59.280635 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 04:34:59.280726 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260806 04:34:59.287931 24364 raft_consensus.cc:1261] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Refusing update from remote peer 04f97d9848aa4dd597e9d6c902cc9ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.288040 23921 raft_consensus.cc:1261] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Refusing update from remote peer 04f97d9848aa4dd597e9d6c902cc9ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.288468 24545 consensus_queue.cc:1035] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.288640 24519 consensus_queue.cc:1035] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.293298 24490 mvcc.cc:205] Tried to move back new op lower bound from 7315418723480207360 to 7315418722260357120. Current Snapshot: MvccSnapshot[applied={T|T < 7315418723480207360}]
I20260806 04:34:59.331907 23670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:59.332082 23670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:59.334321 23670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Sending DeleteTablet for 3 replicas of tablet af9f78a8774846b7a50d1cff1b7455f5
I20260806 04:34:59.334951 24306 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f78a8774846b7a50d1cff1b7455f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:33366
I20260806 04:34:59.334966 23891 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f78a8774846b7a50d1cff1b7455f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:58036
I20260806 04:34:59.335160 24507 tablet_replica.cc:330] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: stopping tablet replica
I20260806 04:34:59.335274 24507 raft_consensus.cc:2229] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:59.335254 24106 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f78a8774846b7a50d1cff1b7455f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47098
I20260806 04:34:59.335390 24507 raft_consensus.cc:2258] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.335453 24503 tablet_replica.cc:330] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: stopping tablet replica
I20260806 04:34:59.335462 24509 tablet_replica.cc:330] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: stopping tablet replica
I20260806 04:34:59.335556 24503 raft_consensus.cc:2229] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:59.335592 24509 raft_consensus.cc:2229] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:59.335719 24509 raft_consensus.cc:2258] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.335860 24503 raft_consensus.cc:2258] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.336606 24507 ts_tablet_manager.cc:1905] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.338975 24503 ts_tablet_manager.cc:1905] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.339716 24507 ts_tablet_manager.cc:1918] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:59.339785 24507 log.cc:1196] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/af9f78a8774846b7a50d1cff1b7455f5
I20260806 04:34:59.340021 24507 ts_tablet_manager.cc:1939] T af9f78a8774846b7a50d1cff1b7455f5 P d5344e20885f4f36b32d68f87cef6fe8: Deleting consensus metadata
I20260806 04:34:59.340817 23647 catalog_manager.cc:4841] TS d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609): tablet af9f78a8774846b7a50d1cff1b7455f5 (table test-table [id=c28e4bf8138743ee8680c9734f11a116]) successfully deleted
I20260806 04:34:59.343401 24503 ts_tablet_manager.cc:1918] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:59.343502 24503 log.cc:1196] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/af9f78a8774846b7a50d1cff1b7455f5
I20260806 04:34:59.343851 24503 ts_tablet_manager.cc:1939] T af9f78a8774846b7a50d1cff1b7455f5 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting consensus metadata
I20260806 04:34:59.344830 23646 catalog_manager.cc:4841] TS 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857): tablet af9f78a8774846b7a50d1cff1b7455f5 (table test-table [id=c28e4bf8138743ee8680c9734f11a116]) successfully deleted
I20260806 04:34:59.345162 24509 ts_tablet_manager.cc:1905] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.347357 24509 ts_tablet_manager.cc:1918] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:59.347424 24509 log.cc:1196] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/af9f78a8774846b7a50d1cff1b7455f5
I20260806 04:34:59.347637 24509 ts_tablet_manager.cc:1939] T af9f78a8774846b7a50d1cff1b7455f5 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting consensus metadata
I20260806 04:34:59.348567 23647 catalog_manager.cc:4841] TS da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527): tablet af9f78a8774846b7a50d1cff1b7455f5 (table test-table [id=c28e4bf8138743ee8680c9734f11a116]) successfully deleted
Aug 06 04:34:59 dist-test-slave-xt4k krb5kdc[23610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 04:34:59 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:34:59 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:34:59.369366 23751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:34:59.362446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45796 (local address 127.12.15.254:33599)
0806 04:34:59.362619 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.362624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.362732 (+ 108us) server_negotiation.cc:407] Connection header received
0806 04:34:59.362853 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.362857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.362917 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.363030 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:59.363825 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.364528 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.365464 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.365850 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.365977 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.369238 (+ 3261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45796: BlockingRecv error: recv got EOF from 127.0.0.1:45796 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 04:34:59 dist-test-slave-xt4k krb5kdc[23610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM
I20260806 04:34:59.379452 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.371014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.12.15.254:33599)
0806 04:34:59.371134 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.371139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.371229 (+ 90us) server_negotiation.cc:407] Connection header received
0806 04:34:59.371347 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.371350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.371396 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.371462 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:34:59.372016 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.372815 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.373575 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.373910 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.374023 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.376204 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:34:59.376208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:34:59.376211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:34:59.376240 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:34:59.377718 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:59.378115 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:59.378119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:59.378121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:59.378233 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:34:59.378526 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:34:59.378528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:34:59.378530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:34:59.378744 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:34:59.378879 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.379143 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.379276 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 04:34:59.382148 23670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:59.382618 23670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:59.387382 23891 tablet_service.cc:1467] Processing CreateTablet for tablet f199b2cda5ca4138aa13e1955be5b237 (DEFAULT_TABLE table=test-table [id=1ed864b0e3cd47b3b042f0346b4e1697]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:59.387567 24306 tablet_service.cc:1467] Processing CreateTablet for tablet f199b2cda5ca4138aa13e1955be5b237 (DEFAULT_TABLE table=test-table [id=1ed864b0e3cd47b3b042f0346b4e1697]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:59.387604 24106 tablet_service.cc:1467] Processing CreateTablet for tablet f199b2cda5ca4138aa13e1955be5b237 (DEFAULT_TABLE table=test-table [id=1ed864b0e3cd47b3b042f0346b4e1697]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:59.387714 23891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f199b2cda5ca4138aa13e1955be5b237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.387847 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f199b2cda5ca4138aa13e1955be5b237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.387847 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f199b2cda5ca4138aa13e1955be5b237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.390090 24470 tablet_bootstrap.cc:492] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Bootstrap starting.
I20260806 04:34:59.390199 24472 tablet_bootstrap.cc:492] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Bootstrap starting.
I20260806 04:34:59.390199 24473 tablet_bootstrap.cc:492] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Bootstrap starting.
I20260806 04:34:59.391008 24473 tablet_bootstrap.cc:654] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.391019 24472 tablet_bootstrap.cc:654] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.391057 24470 tablet_bootstrap.cc:654] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.392378 24470 tablet_bootstrap.cc:492] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: No bootstrap required, opened a new log
I20260806 04:34:59.392467 24470 ts_tablet_manager.cc:1397] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 04:34:59.392782 24472 tablet_bootstrap.cc:492] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: No bootstrap required, opened a new log
I20260806 04:34:59.392859 24472 ts_tablet_manager.cc:1397] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:59.392925 24470 raft_consensus.cc:348] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.393033 24470 raft_consensus.cc:374] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.393064 24470 raft_consensus.cc:729] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5344e20885f4f36b32d68f87cef6fe8, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.393205 24473 tablet_bootstrap.cc:492] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: No bootstrap required, opened a new log
I20260806 04:34:59.393190 24470 consensus_queue.cc:260] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.393283 24473 ts_tablet_manager.cc:1397] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 04:34:59.393239 24472 raft_consensus.cc:348] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.393337 24472 raft_consensus.cc:374] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.393369 24472 raft_consensus.cc:729] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.393443 24470 ts_tablet_manager.cc:1428] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:59.393513 24472 consensus_queue.cc:260] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.393677 24473 raft_consensus.cc:348] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.393772 24472 ts_tablet_manager.cc:1428] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:59.393803 24473 raft_consensus.cc:374] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.393836 24473 raft_consensus.cc:729] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.393955 24473 consensus_queue.cc:260] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.394212 24473 ts_tablet_manager.cc:1428] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:59.507762 24498 raft_consensus.cc:482] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:59.507957 24498 raft_consensus.cc:504] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.508476 24498 leader_election.cc:288] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857)
I20260806 04:34:59.509035 24137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f199b2cda5ca4138aa13e1955be5b237" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" is_pre_election: true
I20260806 04:34:59.509037 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f199b2cda5ca4138aa13e1955be5b237" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8" is_pre_election: true
I20260806 04:34:59.509209 24137 raft_consensus.cc:2454] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 0.
I20260806 04:34:59.509215 23921 raft_consensus.cc:2454] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 0.
I20260806 04:34:59.509522 24248 leader_election.cc:302] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5344e20885f4f36b32d68f87cef6fe8, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:59.509735 24498 raft_consensus.cc:2781] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:59.509806 24498 raft_consensus.cc:482] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:59.509841 24498 raft_consensus.cc:3037] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.511147 24498 raft_consensus.cc:504] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.511503 24498 leader_election.cc:288] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 election: Requested vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857)
I20260806 04:34:59.511888 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f199b2cda5ca4138aa13e1955be5b237" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8"
I20260806 04:34:59.511895 24137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f199b2cda5ca4138aa13e1955be5b237" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2"
I20260806 04:34:59.512018 24137 raft_consensus.cc:3037] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.512033 23921 raft_consensus.cc:3037] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.513181 23921 raft_consensus.cc:2454] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 1.
I20260806 04:34:59.513181 24137 raft_consensus.cc:2454] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 1.
I20260806 04:34:59.513502 24247 leader_election.cc:302] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:59.513676 24498 raft_consensus.cc:2781] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:59.513849 24498 raft_consensus.cc:686] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 LEADER]: Becoming Leader. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Running, Role: LEADER
I20260806 04:34:59.514009 24498 consensus_queue.cc:237] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.515821 23670 catalog_manager.cc:5495] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e reported cstate change: term changed from 0 to 1, leader changed from <none> to da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195). New cstate: current_term: 1 leader_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } health_report { overall_health: HEALTHY } } }
I20260806 04:34:59.552430 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.549570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.12.15.195:44527)
0806 04:34:59.549802 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.549814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.549826 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:34:59.549870 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.549872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.549916 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.549989 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:59.550181 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.550720 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.551527 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.551753 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.551833 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.552042 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.552110 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.552163 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.552224 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.552271 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":78,"threads_started":1}
I20260806 04:34:59.554414 24137 raft_consensus.cc:1261] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Refusing update from remote peer da55e449b8d94f33bff4a3a2db57fd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.554423 23921 raft_consensus.cc:1261] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Refusing update from remote peer da55e449b8d94f33bff4a3a2db57fd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.554836 24579 consensus_queue.cc:1035] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.555002 24498 consensus_queue.cc:1035] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.561816 23670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:59.561931 23670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:59.563791 23670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Sending DeleteTablet for 3 replicas of tablet f199b2cda5ca4138aa13e1955be5b237
I20260806 04:34:59.564292 23891 tablet_service.cc:1514] Processing DeleteTablet for tablet f199b2cda5ca4138aa13e1955be5b237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:58036
I20260806 04:34:59.564388 24106 tablet_service.cc:1514] Processing DeleteTablet for tablet f199b2cda5ca4138aa13e1955be5b237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47098
I20260806 04:34:59.564507 24306 tablet_service.cc:1514] Processing DeleteTablet for tablet f199b2cda5ca4138aa13e1955be5b237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:33366
I20260806 04:34:59.564574 24503 tablet_replica.cc:330] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: stopping tablet replica
I20260806 04:34:59.564684 24509 tablet_replica.cc:330] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: stopping tablet replica
I20260806 04:34:59.564687 24503 raft_consensus.cc:2229] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:59.564694 24507 tablet_replica.cc:330] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: stopping tablet replica
I20260806 04:34:59.564790 24509 raft_consensus.cc:2229] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:59.564806 24503 raft_consensus.cc:2258] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.564805 24507 raft_consensus.cc:2229] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:59.564921 24507 raft_consensus.cc:2258] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.564963 24509 raft_consensus.cc:2258] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.565133 23670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:34:59.565521 23670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:34:59.565657 24507 ts_tablet_manager.cc:1905] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.565740 24503 ts_tablet_manager.cc:1905] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.567824 24503 ts_tablet_manager.cc:1918] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:59.567902 24503 log.cc:1196] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/f199b2cda5ca4138aa13e1955be5b237
I20260806 04:34:59.568109 24507 ts_tablet_manager.cc:1918] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:59.568161 24503 ts_tablet_manager.cc:1939] T f199b2cda5ca4138aa13e1955be5b237 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting consensus metadata
I20260806 04:34:59.568182 24507 log.cc:1196] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/f199b2cda5ca4138aa13e1955be5b237
I20260806 04:34:59.568493 24507 ts_tablet_manager.cc:1939] T f199b2cda5ca4138aa13e1955be5b237 P d5344e20885f4f36b32d68f87cef6fe8: Deleting consensus metadata
I20260806 04:34:59.569198 23647 catalog_manager.cc:4841] TS d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609): tablet f199b2cda5ca4138aa13e1955be5b237 (table test-table [id=1ed864b0e3cd47b3b042f0346b4e1697]) successfully deleted
I20260806 04:34:59.569548 24509 ts_tablet_manager.cc:1905] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.569715 23646 catalog_manager.cc:4841] TS 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857): tablet f199b2cda5ca4138aa13e1955be5b237 (table test-table [id=1ed864b0e3cd47b3b042f0346b4e1697]) successfully deleted
I20260806 04:34:59.570070 23891 tablet_service.cc:1467] Processing CreateTablet for tablet 3c06d859c2f2445aabde3096820f7e83 (DEFAULT_TABLE table=test-table [id=8dcd9f2c461a4b6592f05878a8f895ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:59.570209 24106 tablet_service.cc:1467] Processing CreateTablet for tablet 3c06d859c2f2445aabde3096820f7e83 (DEFAULT_TABLE table=test-table [id=8dcd9f2c461a4b6592f05878a8f895ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:59.570310 23891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c06d859c2f2445aabde3096820f7e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.570439 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c06d859c2f2445aabde3096820f7e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.571933 24509 ts_tablet_manager.cc:1918] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:34:59.572019 24509 log.cc:1196] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/f199b2cda5ca4138aa13e1955be5b237
I20260806 04:34:59.572013 24470 tablet_bootstrap.cc:492] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: Bootstrap starting.
I20260806 04:34:59.572271 24509 ts_tablet_manager.cc:1939] T f199b2cda5ca4138aa13e1955be5b237 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting consensus metadata
I20260806 04:34:59.572598 24473 tablet_bootstrap.cc:492] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Bootstrap starting.
I20260806 04:34:59.573205 24470 tablet_bootstrap.cc:654] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.573323 24473 tablet_bootstrap.cc:654] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.573899 24306 tablet_service.cc:1467] Processing CreateTablet for tablet 3c06d859c2f2445aabde3096820f7e83 (DEFAULT_TABLE table=test-table [id=8dcd9f2c461a4b6592f05878a8f895ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:34:59.574124 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c06d859c2f2445aabde3096820f7e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:34:59.574393 24473 tablet_bootstrap.cc:492] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: No bootstrap required, opened a new log
I20260806 04:34:59.574488 24473 ts_tablet_manager.cc:1397] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:59.574870 24470 tablet_bootstrap.cc:492] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: No bootstrap required, opened a new log
I20260806 04:34:59.574944 24470 ts_tablet_manager.cc:1397] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:34:59.574879 24473 raft_consensus.cc:348] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.574985 24473 raft_consensus.cc:374] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.575039 24473 raft_consensus.cc:729] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f97d9848aa4dd597e9d6c902cc9ee2, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.575147 24473 consensus_queue.cc:260] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.575387 24473 ts_tablet_manager.cc:1428] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 04:34:59.575359 24470 raft_consensus.cc:348] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.575456 24470 raft_consensus.cc:374] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.575486 24470 raft_consensus.cc:729] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5344e20885f4f36b32d68f87cef6fe8, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.575610 24470 consensus_queue.cc:260] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.575852 24470 ts_tablet_manager.cc:1428] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:34:59.576273 24472 tablet_bootstrap.cc:492] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Bootstrap starting.
I20260806 04:34:59.576942 24472 tablet_bootstrap.cc:654] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Neither blocks nor log segments found. Creating new log.
I20260806 04:34:59.577533 23647 catalog_manager.cc:4841] TS da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527): tablet f199b2cda5ca4138aa13e1955be5b237 (table test-table [id=1ed864b0e3cd47b3b042f0346b4e1697]) successfully deleted
I20260806 04:34:59.578176 24472 tablet_bootstrap.cc:492] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: No bootstrap required, opened a new log
I20260806 04:34:59.578261 24472 ts_tablet_manager.cc:1397] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 04:34:59.578599 24472 raft_consensus.cc:348] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.578683 24472 raft_consensus.cc:374] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:34:59.578703 24472 raft_consensus.cc:729] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Initialized, Role: FOLLOWER
I20260806 04:34:59.578816 24472 consensus_queue.cc:260] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.579041 24472 ts_tablet_manager.cc:1428] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 04:34:59.851742 24588 raft_consensus.cc:482] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:34:59.851897 24588 raft_consensus.cc:504] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.852385 24588 leader_election.cc:288] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857)
I20260806 04:34:59.852841 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c06d859c2f2445aabde3096820f7e83" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8" is_pre_election: true
I20260806 04:34:59.852849 24137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c06d859c2f2445aabde3096820f7e83" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" is_pre_election: true
I20260806 04:34:59.852993 23921 raft_consensus.cc:2454] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 0.
I20260806 04:34:59.852994 24137 raft_consensus.cc:2454] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 0.
I20260806 04:34:59.853295 24248 leader_election.cc:302] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5344e20885f4f36b32d68f87cef6fe8, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:59.853477 24588 raft_consensus.cc:2781] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:34:59.853536 24588 raft_consensus.cc:482] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:34:59.853564 24588 raft_consensus.cc:3037] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.854609 24588 raft_consensus.cc:504] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.854950 24588 leader_election.cc:288] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 election: Requested vote from peers d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609), 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857)
I20260806 04:34:59.855311 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c06d859c2f2445aabde3096820f7e83" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5344e20885f4f36b32d68f87cef6fe8"
I20260806 04:34:59.855340 24137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c06d859c2f2445aabde3096820f7e83" candidate_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2"
I20260806 04:34:59.855448 24137 raft_consensus.cc:3037] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.855468 23921 raft_consensus.cc:3037] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:34:59.856371 24137 raft_consensus.cc:2454] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 1.
I20260806 04:34:59.856397 23921 raft_consensus.cc:2454] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da55e449b8d94f33bff4a3a2db57fd6e in term 1.
I20260806 04:34:59.856650 24247 leader_election.cc:302] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f97d9848aa4dd597e9d6c902cc9ee2, da55e449b8d94f33bff4a3a2db57fd6e; no voters:
I20260806 04:34:59.856822 24588 raft_consensus.cc:2781] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:34:59.856914 24588 raft_consensus.cc:686] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 LEADER]: Becoming Leader. State: Replica: da55e449b8d94f33bff4a3a2db57fd6e, State: Running, Role: LEADER
I20260806 04:34:59.857056 24588 consensus_queue.cc:237] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } }
I20260806 04:34:59.858733 23674 catalog_manager.cc:5495] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e reported cstate change: term changed from 0 to 1, leader changed from <none> to da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195). New cstate: current_term: 1 leader_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da55e449b8d94f33bff4a3a2db57fd6e" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44527 } health_report { overall_health: HEALTHY } } }
I20260806 04:34:59.883538 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.880098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:44889 (local address 127.12.15.194:37857)
0806 04:34:59.880311 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.880317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.880376 (+ 59us) server_negotiation.cc:407] Connection header received
0806 04:34:59.880591 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.880595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.880646 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.880716 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:59.880953 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.881490 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.882332 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.882599 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.882690 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.882969 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.883051 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.883109 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.883213 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.883266 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":61,"threads_started":1}
I20260806 04:34:59.883517 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.879955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:37857 (local address 127.12.15.195:44889)
0806 04:34:59.880279 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:59.880296 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:59.880468 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:59.880740 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:59.880743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:59.880752 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:59.880856 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.880861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.881606 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.881610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.882223 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.882232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.882715 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.882719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.882797 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.882841 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.883130 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.883137 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:34:59.883195 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 04:34:59.883278 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260806 04:34:59.889658 24137 raft_consensus.cc:1261] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Refusing update from remote peer da55e449b8d94f33bff4a3a2db57fd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.889657 23921 raft_consensus.cc:1261] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Refusing update from remote peer da55e449b8d94f33bff4a3a2db57fd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:34:59.890028 24498 consensus_queue.cc:1035] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5344e20885f4f36b32d68f87cef6fe8" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.890177 24588 consensus_queue.cc:1035] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "04f97d9848aa4dd597e9d6c902cc9ee2" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:34:59.907982 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.904665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33053 (local address 127.12.15.195:44527)
0806 04:34:59.904776 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 04:34:59.904779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:34:59.904960 (+ 181us) server_negotiation.cc:407] Connection header received
0806 04:34:59.905064 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:34:59.905066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:34:59.905106 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:34:59.905163 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:34:59.905314 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.905809 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.906883 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.907177 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.907271 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.907569 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.907666 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.907737 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 04:34:59.907830 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 04:34:59.907876 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 04:34:59.908007 24609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.904619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:44527 (local address 127.12.15.194:33053)
0806 04:34:59.904901 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:59.904913 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:34:59.904998 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:34:59.905167 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:34:59.905169 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:34:59.905174 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 04:34:59.905259 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.905263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.905927 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.905931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.906779 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:34:59.906787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:34:59.907310 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:34:59.907314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:34:59.907410 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:34:59.907444 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 04:34:59.907763 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 04:34:59.907770 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:34:59.907821 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 04:34:59.907895 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260806 04:34:59.925405 23670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:59.925563 23670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:34:59.927518 23670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 419b86a04b654b33aee8ee75f12cb58f: Sending DeleteTablet for 3 replicas of tablet 3c06d859c2f2445aabde3096820f7e83
I20260806 04:34:59.928047 23891 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c06d859c2f2445aabde3096820f7e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:58036
I20260806 04:34:59.928128 24306 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c06d859c2f2445aabde3096820f7e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:33366
I20260806 04:34:59.928249 24507 tablet_replica.cc:330] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: stopping tablet replica
I20260806 04:34:59.928252 24106 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c06d859c2f2445aabde3096820f7e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:34:59 UTC) from {username='oryx', principal='oryx/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47098
I20260806 04:34:59.928305 24509 tablet_replica.cc:330] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: stopping tablet replica
I20260806 04:34:59.928385 24507 raft_consensus.cc:2229] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:59.928445 24503 tablet_replica.cc:330] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: stopping tablet replica
I20260806 04:34:59.928436 24509 raft_consensus.cc:2229] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:34:59.928494 24507 raft_consensus.cc:2258] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.928539 24503 raft_consensus.cc:2229] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:34:59.928623 24509 raft_consensus.cc:2258] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.928645 24503 raft_consensus.cc:2258] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:34:59.929280 24507 ts_tablet_manager.cc:1905] T 3c06d859c2f2445aabde3096820f7e83 P d5344e20885f4f36b32d68f87cef6fe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.929347 24509 ts_tablet_manager.cc:1905] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.929347 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 23789
I20260806 04:34:59.929386 24503 ts_tablet_manager.cc:1905] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:34:59.931411 24509 ts_tablet_manager.cc:1918] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:59.931465 24509 log.cc:1196] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/3c06d859c2f2445aabde3096820f7e83
I20260806 04:34:59.931648 24509 ts_tablet_manager.cc:1939] T 3c06d859c2f2445aabde3096820f7e83 P da55e449b8d94f33bff4a3a2db57fd6e: Deleting consensus metadata
I20260806 04:34:59.932221 24503 ts_tablet_manager.cc:1918] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 04:34:59.932288 24503 log.cc:1196] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/3c06d859c2f2445aabde3096820f7e83
I20260806 04:34:59.932255 23647 catalog_manager.cc:4841] TS da55e449b8d94f33bff4a3a2db57fd6e (127.12.15.195:44527): tablet 3c06d859c2f2445aabde3096820f7e83 (table test-table [id=8dcd9f2c461a4b6592f05878a8f895ea]) successfully deleted
I20260806 04:34:59.933012 24503 ts_tablet_manager.cc:1939] T 3c06d859c2f2445aabde3096820f7e83 P 04f97d9848aa4dd597e9d6c902cc9ee2: Deleting consensus metadata
I20260806 04:34:59.935411 23646 catalog_manager.cc:4841] TS 04f97d9848aa4dd597e9d6c902cc9ee2 (127.12.15.194:37857): tablet 3c06d859c2f2445aabde3096820f7e83 (table test-table [id=8dcd9f2c461a4b6592f05878a8f895ea]) successfully deleted
W20260806 04:34:59.942744 23647 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:46609 (error 108)
I20260806 04:34:59.943234 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 24007
I20260806 04:34:59.948690 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:34:59.943455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:46609 (local address 127.0.0.1:58046)
0806 04:34:59.948503 (+ 5048us) negotiation.cc:107] Waiting for socket to connect
0806 04:34:59.948585 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4962,"thread_start_us":53,"threads_started":1}
W20260806 04:34:59.948804 23647 catalog_manager.cc:4584] TS d5344e20885f4f36b32d68f87cef6fe8 (127.12.15.193:46609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c06d859c2f2445aabde3096820f7e83: Network error: Client connection negotiation failed: client connection to 127.12.15.193:46609: connect: Connection refused (error 111)
I20260806 04:34:59.958992 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 24221
I20260806 04:34:59.974915 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 23626
2026-08-06T04:34:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3474 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:35:00 dist-test-slave-xt4k krb5kdc[24618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:35:00 dist-test-slave-xt4k krb5kdc[24618](info): set up 2 sockets
Aug 06 04:35:00 dist-test-slave-xt4k krb5kdc[24618](info): commencing operation
krb5kdc: starting...
W20260806 04:35:01.116798 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:35:01 dist-test-slave-xt4k krb5kdc[24618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:35:01.244757 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43837
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:43837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:01.358726 24636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:01.358982 24636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:01.359021 24636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:01.363013 24636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:35:01.363071 24636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:01.363117 24636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:01.363138 24636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:35:01.363165 24636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:35:01.368541 24636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:43837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:01.369576 24636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:01.370569 24636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:01.378846 24642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:01.378896 24636 server_base.cc:1034] running on GCE node
W20260806 04:35:01.378901 24649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:01.379072 24643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:01.379621 24636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:01.380761 24636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:01.381927 24636 hybrid_clock.cc:648] HybridClock initialized: now 1785990901381906 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T04:35:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1412 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:35:01 dist-test-slave-xt4k krb5kdc[24658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:35:01 dist-test-slave-xt4k krb5kdc[24658](info): set up 2 sockets
Aug 06 04:35:01 dist-test-slave-xt4k krb5kdc[24658](info): commencing operation
krb5kdc: starting...
W20260806 04:35:02.527801 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:35:02 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:35:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:35:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:35:02.654870 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34299
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:34299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:02.769706 24679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:02.769970 24679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:02.770025 24679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:02.774112 24679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:35:02.774170 24679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:02.774196 24679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:02.774219 24679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:35:02.774248 24679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:35:02.779516 24679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:34299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:02.780580 24679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:02.781576 24679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:02.789600 24692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:02.789594 24689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:02.789635 24690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:02.789752 24679 server_base.cc:1034] running on GCE node
I20260806 04:35:02.790387 24679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:02.791406 24679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:02.792565 24679 hybrid_clock.cc:648] HybridClock initialized: now 1785990902792550 us; error 27 us; skew 500 ppm
Aug 06 04:35:02 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990902, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:02.798448 24679 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:35:02.799450 24679 webserver.cc:460] Webserver started at http://127.12.15.254:44089/ using document root <none> and password file <none>
I20260806 04:35:02.800158 24679 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:02.800251 24679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:02.800532 24679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:02.802939 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "7c00cf552d7b4eb09b3c79623eb72451"
format_stamp: "Formatted at 2026-08-06 04:35:02 on dist-test-slave-xt4k"
server_key: "1e2bf788ff6941b921c9d2c9f6022dcb"
server_key_iv: "bb64a920b8c39e0a4e0d36c421b9993b"
server_key_version: "encryptionkey@0"
I20260806 04:35:02.803444 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "7c00cf552d7b4eb09b3c79623eb72451"
format_stamp: "Formatted at 2026-08-06 04:35:02 on dist-test-slave-xt4k"
server_key: "1e2bf788ff6941b921c9d2c9f6022dcb"
server_key_iv: "bb64a920b8c39e0a4e0d36c421b9993b"
server_key_version: "encryptionkey@0"
I20260806 04:35:02.807171 24679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 04:35:02.809556 24701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:02.810712 24679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:35:02.810817 24679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "7c00cf552d7b4eb09b3c79623eb72451"
format_stamp: "Formatted at 2026-08-06 04:35:02 on dist-test-slave-xt4k"
server_key: "1e2bf788ff6941b921c9d2c9f6022dcb"
server_key_iv: "bb64a920b8c39e0a4e0d36c421b9993b"
server_key_version: "encryptionkey@0"
I20260806 04:35:02.810928 24679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:02.833283 24679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:02.836419 24679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:02.836611 24679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:02.843724 24679 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:34299
I20260806 04:35:02.843742 24798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:34299 every 8 connection(s)
I20260806 04:35:02.844759 24679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:35:02.847549 24801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:02.851756 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 24679
I20260806 04:35:02.851893 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:35:02.852188 12351 external_mini_cluster.cc:1428] Setting key 3401dda2d5436b930be3f8e3dc2807e1
I20260806 04:35:02.853578 24801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Bootstrap starting.
I20260806 04:35:02.856014 24801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:02.856829 24801 log.cc:824] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:02.858889 24801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: No bootstrap required, opened a new log
Aug 06 04:35:02 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:35:02.863221 24801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:02.863492 24801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:02.863550 24801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c00cf552d7b4eb09b3c79623eb72451, State: Initialized, Role: FOLLOWER
I20260806 04:35:02.864017 24801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:02.864163 24801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:35:02.864238 24801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:35:02.864358 24801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:02.865505 24801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:02.865870 24801 leader_election.cc:302] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c00cf552d7b4eb09b3c79623eb72451; no voters:
I20260806 04:35:02.866256 24801 leader_election.cc:288] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:35:02.866348 24811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:35:02.866559 24811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 LEADER]: Becoming Leader. State: Replica: 7c00cf552d7b4eb09b3c79623eb72451, State: Running, Role: LEADER
I20260806 04:35:02.866873 24811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:02.866938 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:02.853623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42076 (local address 127.12.15.254:34299)
0806 04:35:02.854036 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:02.854052 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:02.854088 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:35:02.854737 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:02.854754 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:02.855074 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:02.855348 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:02.856245 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:02.857335 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:02.858220 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:02.858924 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:02.859088 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:02.861645 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:02.861653 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:02.861673 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:02.861711 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:02.863723 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:02.864145 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:02.864150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:02.864157 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:02.864299 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:02.864619 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:02.864622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:02.864624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:02.865131 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:02.865340 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:02.865616 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 04:35:02.865923 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":103,"threads_started":1}
I20260806 04:35:02.867527 24801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:35:02.868541 24815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c00cf552d7b4eb09b3c79623eb72451. Latest consensus state: current_term: 1 leader_uuid: "7c00cf552d7b4eb09b3c79623eb72451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } } }
I20260806 04:35:02.868667 24815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:02.868902 24812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c00cf552d7b4eb09b3c79623eb72451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } } }
I20260806 04:35:02.868978 24812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:02.869586 24824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:35:02.872783 24824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:35:02.878366 24824 catalog_manager.cc:1347] Generated new cluster ID: 3adeaa313b4e486dba307af298eb8bf5
I20260806 04:35:02.878451 24824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:35:02.900236 24824 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:35:02.901262 24824 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:35:02.906961 24824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Generated new TSK 0
I20260806 04:35:02.907786 24824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:35:02.949856 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34299
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:03.070267 24841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:03.070521 24841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:03.070561 24841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:03.074481 24841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:03.074539 24841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:03.074596 24841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:35:03.079785 24841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:03.080854 24841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:03.081871 24841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:03.092097 24848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:03.092167 24849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:03.092100 24851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:03.092139 24841 server_base.cc:1034] running on GCE node
I20260806 04:35:03.092782 24841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:03.093438 24841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:03.094617 24841 hybrid_clock.cc:648] HybridClock initialized: now 1785990903094588 us; error 40 us; skew 500 ppm
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:03.097476 24841 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:35:03.098349 24841 webserver.cc:460] Webserver started at http://127.12.15.193:44987/ using document root <none> and password file <none>
I20260806 04:35:03.098902 24841 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:03.098969 24841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:03.099186 24841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:03.101336 24841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "4af1363dc62b4fd1a10263de87715538"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "08a8c89724bd4557cb6d6f294f2b28cb"
server_key_iv: "ae43a04bea4c9a97e11a636b9652a4e3"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.101886 24841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "4af1363dc62b4fd1a10263de87715538"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "08a8c89724bd4557cb6d6f294f2b28cb"
server_key_iv: "ae43a04bea4c9a97e11a636b9652a4e3"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.105588 24841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:35:03.108067 24865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.109354 24841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:35:03.109483 24841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "4af1363dc62b4fd1a10263de87715538"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "08a8c89724bd4557cb6d6f294f2b28cb"
server_key_iv: "ae43a04bea4c9a97e11a636b9652a4e3"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.109578 24841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:03.139549 24841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:03.142843 24841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:03.143035 24841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:03.143680 24841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:03.144556 24841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:03.144605 24841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.144660 24841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:03.144685 24841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.153231 24841 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:35603
I20260806 04:35:03.153271 25045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:35603 every 8 connection(s)
I20260806 04:35:03.154129 24841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:35:03.158097 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 24841
I20260806 04:35:03.158232 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:35:03.158715 12351 external_mini_cluster.cc:1428] Setting key 2282e2bd0e976f7de1474503650102e1
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:35:03.169251 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:03.155943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49721 (local address 127.12.15.254:34299)
0806 04:35:03.156080 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:03.156084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:03.156966 (+ 882us) server_negotiation.cc:407] Connection header received
0806 04:35:03.158484 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:03.158487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:03.158545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:03.158610 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:03.160186 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.160863 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.162528 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.162781 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.162888 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:03.165702 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:03.165707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:03.165709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:03.165742 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:03.167150 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:03.167639 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:03.167643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:03.167644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:03.167725 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:03.168053 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:03.168056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:03.168058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:03.168272 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:03.168400 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:03.168906 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 04:35:03.169053 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:35:03.170063 25053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:03.156297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:49721)
0806 04:35:03.156804 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:03.156847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:03.158145 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:03.158825 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:03.158835 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:03.159287 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:03.159919 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:03.159941 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.161024 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.161028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:03.162336 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:03.162350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.162953 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.162957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:03.163137 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:03.163798 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:03.163817 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:03.165508 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:03.167259 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:03.167265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:03.167282 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:03.167537 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:03.167808 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:03.167811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:03.167813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:03.167954 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:03.168401 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:03.168408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:03.168671 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 04:35:03.168848 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 04:35:03.169201 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":117,"threads_started":1}
I20260806 04:35:03.171437 25046 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34299
I20260806 04:35:03.171696 25046 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:35:03.172323 25046 heartbeater.cc:511] Master 127.12.15.254:34299 requested a full tablet report, sending...
I20260806 04:35:03.173995 24730 ts_manager.cc:194] Registered new tserver with Master: 4af1363dc62b4fd1a10263de87715538 (127.12.15.193:35603)
I20260806 04:35:03.175495 24730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:49721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:35:03.189666 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34299
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:03.302167 25058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:03.302417 25058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:03.302464 25058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:03.306381 25058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:03.306435 25058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:03.306496 25058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:35:03.311568 25058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:03.312605 25058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:03.313602 25058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:35:03.323338 25058 server_base.cc:1034] running on GCE node
W20260806 04:35:03.323297 25069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:03.323314 25068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:03.323297 25074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:03.323982 25058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:03.324656 25058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:03.325835 25058 hybrid_clock.cc:648] HybridClock initialized: now 1785990903325817 us; error 29 us; skew 500 ppm
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:03.328588 25058 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:35:03.329475 25058 webserver.cc:460] Webserver started at http://127.12.15.194:40929/ using document root <none> and password file <none>
I20260806 04:35:03.330035 25058 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:03.330106 25058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:03.330312 25058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:03.332389 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "ce25e926c3694ff685bc0e518be52065"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "1fa4c34f801ba7d72de91a8055a88cb9"
server_key_iv: "cd673109cc74548edc810ad83d877f94"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.332938 25058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "ce25e926c3694ff685bc0e518be52065"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "1fa4c34f801ba7d72de91a8055a88cb9"
server_key_iv: "cd673109cc74548edc810ad83d877f94"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.336583 25058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:35:03.338980 25085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.340201 25058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:35:03.340322 25058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "ce25e926c3694ff685bc0e518be52065"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "1fa4c34f801ba7d72de91a8055a88cb9"
server_key_iv: "cd673109cc74548edc810ad83d877f94"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.340476 25058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:03.352133 25058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:03.355012 25058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:03.355193 25058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:03.355778 25058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:03.356618 25058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:03.356666 25058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.356719 25058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:03.356746 25058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.365167 25058 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:35323
I20260806 04:35:03.365197 25263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:35323 every 8 connection(s)
I20260806 04:35:03.366072 25058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:35:03.366390 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 25058
I20260806 04:35:03.366511 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:35:03.366851 12351 external_mini_cluster.cc:1428] Setting key 358ee965aa318dfd07c330aa7f82a693
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:35:03.381227 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:03.368173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:35193 (local address 127.12.15.254:34299)
0806 04:35:03.368457 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:03.368462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:03.369345 (+ 883us) server_negotiation.cc:407] Connection header received
0806 04:35:03.370976 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:03.370980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:03.371032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:03.371115 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:03.372667 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.373222 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.374391 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.374742 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.374854 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:03.377452 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:03.377457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:03.377459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:03.377490 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:03.378998 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:03.379513 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:03.379517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:03.379518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:03.379605 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:03.380000 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:03.380004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:03.380005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:03.380268 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:03.380386 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:03.380931 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 04:35:03.381080 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:35:03.382192 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:03.368605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:35193)
0806 04:35:03.369182 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:03.369237 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:03.370739 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:03.371286 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:03.371295 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:03.371809 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:03.372517 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:03.372537 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.373351 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.373354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:03.374230 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:03.374240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.374894 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.374898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:03.375043 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:03.375641 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:03.375658 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:03.377278 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:03.379109 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:03.379115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:03.379127 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:03.379382 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:03.379716 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:03.379752 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:03.379753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:03.379892 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:03.380399 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:03.380404 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:03.380664 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 04:35:03.380838 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 04:35:03.381183 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":162,"threads_started":1}
I20260806 04:35:03.383399 25264 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34299
I20260806 04:35:03.383636 25264 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:03.384259 25264 heartbeater.cc:511] Master 127.12.15.254:34299 requested a full tablet report, sending...
I20260806 04:35:03.385529 24730 ts_manager.cc:194] Registered new tserver with Master: ce25e926c3694ff685bc0e518be52065 (127.12.15.194:35323)
I20260806 04:35:03.386283 24730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:35193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:35:03.398862 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34299
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:03.513177 25272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:03.513429 25272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:03.513471 25272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:03.517545 25272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:03.517606 25272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:03.517661 25272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:35:03.522955 25272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:34299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:03.523974 25272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:03.525018 25272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:03.534545 25283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:03.534657 25272 server_base.cc:1034] running on GCE node
W20260806 04:35:03.534732 25284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:03.534559 25287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:03.535143 25272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:03.535753 25272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:03.536906 25272 hybrid_clock.cc:648] HybridClock initialized: now 1785990903536888 us; error 39 us; skew 500 ppm
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:03.539610 25272 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:35:03.540472 25272 webserver.cc:460] Webserver started at http://127.12.15.195:44631/ using document root <none> and password file <none>
I20260806 04:35:03.541013 25272 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:03.541077 25272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:03.541281 25272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:03.543372 25272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "84c8bfcf964d4798805404a06548e7d6"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "0ee34a6e4bb38df027b06e012aa20d9b"
server_key_iv: "c4c07ae0b60b7fea7c2d9db5621e9887"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.543913 25272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "84c8bfcf964d4798805404a06548e7d6"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "0ee34a6e4bb38df027b06e012aa20d9b"
server_key_iv: "c4c07ae0b60b7fea7c2d9db5621e9887"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.547770 25272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 04:35:03.550292 25300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.551477 25272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:35:03.551615 25272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "84c8bfcf964d4798805404a06548e7d6"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "0ee34a6e4bb38df027b06e012aa20d9b"
server_key_iv: "c4c07ae0b60b7fea7c2d9db5621e9887"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.551715 25272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:03.571161 25272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:03.574050 25272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:03.574234 25272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:03.574837 25272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:03.575676 25272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:03.575724 25272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.575778 25272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:03.575804 25272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.584148 25272 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39697
I20260806 04:35:03.584172 25474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39697 every 8 connection(s)
I20260806 04:35:03.585032 25272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:35:03.586222 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 25272
I20260806 04:35:03.586355 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:35:03.586710 12351 external_mini_cluster.cc:1428] Setting key 24c960446199a7da0d9a442b008827b1
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:35:03.598469 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:03.586906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33317 (local address 127.12.15.254:34299)
0806 04:35:03.587039 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:03.587042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:03.588164 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 04:35:03.589617 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:03.589620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:03.589673 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:03.589741 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:03.590912 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.591431 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.592570 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.592794 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.592872 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:03.595121 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:03.595124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:03.595126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:03.595152 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:03.596480 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:03.596975 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:03.596979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:03.596981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:03.597060 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:03.597386 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:03.597389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:03.597391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:03.597580 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:03.597673 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:03.598167 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 04:35:03.598296 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 04:35:03.599407 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:03.587431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:33317)
0806 04:35:03.588006 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:03.588059 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:03.589402 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:03.589937 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:03.589944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:03.590288 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:03.590798 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:03.590811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.591556 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.591559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:03.592467 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:03.592477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:03.592906 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:03.592909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:03.593042 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:03.593570 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:03.593585 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:03.594962 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:03.596587 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:03.596594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:03.596610 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:03.596878 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:03.597151 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:03.597154 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:03.597156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:03.597297 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:03.597682 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:03.597688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:03.597935 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 04:35:03.598107 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 04:35:03.598445 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":205,"threads_started":1}
I20260806 04:35:03.600641 25475 heartbeater.cc:347] Connected to a master server at 127.12.15.254:34299
I20260806 04:35:03.600903 25475 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:03.601411 25475 heartbeater.cc:511] Master 127.12.15.254:34299 requested a full tablet report, sending...
I20260806 04:35:03.602478 24730 ts_manager.cc:194] Registered new tserver with Master: 84c8bfcf964d4798805404a06548e7d6 (127.12.15.195:39697)
I20260806 04:35:03.603055 24730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:33317
I20260806 04:35:03.612648 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab.
Entry for principal oryx/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab.
WARNING: no policy specified for oryx/127.12.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab.
Entry for principal oryx/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab.
WARNING: no policy specified for oryx/127.12.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab.
Entry for principal oryx/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab.
I20260806 04:35:03.683563 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 24841
I20260806 04:35:03.696933 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 25058
I20260806 04:35:03.711239 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 25272
I20260806 04:35:03.729135 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 24679
I20260806 04:35:03.737206 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34299
--webserver_interface=127.12.15.254
--webserver_port=44089
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:34299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:03.849445 25491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:03.849731 25491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:03.849787 25491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:03.853780 25491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:35:03.853843 25491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:03.853871 25491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:03.853893 25491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:35:03.853922 25491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:35:03.859270 25491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:34299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=44089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:03.860356 25491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:03.861398 25491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:03.870074 25504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:03.870101 25499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:03.870159 25491 server_base.cc:1034] running on GCE node
W20260806 04:35:03.870101 25500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:03.870872 25491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:03.871946 25491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:03.873117 25491 hybrid_clock.cc:648] HybridClock initialized: now 1785990903873099 us; error 32 us; skew 500 ppm
Aug 06 04:35:03 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990903, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:03.876035 25491 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:35:03.877104 25491 webserver.cc:460] Webserver started at http://127.12.15.254:44089/ using document root <none> and password file <none>
I20260806 04:35:03.877661 25491 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:03.877736 25491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:03.881630 25491 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:35:03.883970 25515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:03.885440 25491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:35:03.885555 25491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "7c00cf552d7b4eb09b3c79623eb72451"
format_stamp: "Formatted at 2026-08-06 04:35:02 on dist-test-slave-xt4k"
server_key: "1e2bf788ff6941b921c9d2c9f6022dcb"
server_key_iv: "bb64a920b8c39e0a4e0d36c421b9993b"
server_key_version: "encryptionkey@0"
I20260806 04:35:03.886117 25491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:03.901574 25491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:03.904635 25491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:03.904795 25491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:03.911607 25491 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:34299
I20260806 04:35:03.911666 25596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:34299 every 8 connection(s)
I20260806 04:35:03.912724 25491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:35:03.913779 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 25491
I20260806 04:35:03.914199 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:35603
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:34299
--webserver_port=44987
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:35:03.917971 25597 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:35:03.919874 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Bootstrap starting.
I20260806 04:35:03.937141 25597 log.cc:824] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:03.944327 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:35:03.944814 25597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Bootstrap complete.
I20260806 04:35:03.949082 25597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:03.949553 25597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c00cf552d7b4eb09b3c79623eb72451, State: Initialized, Role: FOLLOWER
I20260806 04:35:03.950088 25597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:03.950214 25597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:35:03.950282 25597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:35:03.950376 25597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:35:03.952375 25597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:03.952721 25597 leader_election.cc:302] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c00cf552d7b4eb09b3c79623eb72451; no voters:
I20260806 04:35:03.953050 25597 leader_election.cc:288] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 04:35:03.953181 25602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 04:35:03.953382 25602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [term 2 LEADER]: Becoming Leader. State: Replica: 7c00cf552d7b4eb09b3c79623eb72451, State: Running, Role: LEADER
I20260806 04:35:03.953691 25602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } }
I20260806 04:35:03.954355 25597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:35:03.954701 25602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c00cf552d7b4eb09b3c79623eb72451. Latest consensus state: current_term: 2 leader_uuid: "7c00cf552d7b4eb09b3c79623eb72451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } } }
I20260806 04:35:03.954820 25602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:03.955294 25607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:35:03.955267 25603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7c00cf552d7b4eb09b3c79623eb72451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c00cf552d7b4eb09b3c79623eb72451" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34299 } } }
I20260806 04:35:03.955488 25603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:03.958761 25607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:35:03.959434 25607 catalog_manager.cc:1259] Loaded cluster ID: 3adeaa313b4e486dba307af298eb8bf5
I20260806 04:35:03.959589 25607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:35:03.962062 25607 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:35:03.962809 25607 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7c00cf552d7b4eb09b3c79623eb72451: Loaded TSK: 0
I20260806 04:35:03.964100 25607 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:35:04.036742 25600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:04.036996 25600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:04.037065 25600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:04.041096 25600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:04.041155 25600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:04.041231 25600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:35:04.046471 25600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.193.keytab
--principal=oryx/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:35603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=44987
--tserver_master_addrs=127.12.15.254:34299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:04.047415 25600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:04.048421 25600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:04.057921 25633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:04.058097 25635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:04.057969 25632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:04.058302 25600 server_base.cc:1034] running on GCE node
I20260806 04:35:04.058691 25600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:04.059286 25600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:04.060421 25600 hybrid_clock.cc:648] HybridClock initialized: now 1785990904060383 us; error 54 us; skew 500 ppm
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990904, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:04.063176 25600 init.cc:377] Logged in from keytab as oryx/127.12.15.193@KRBTEST.COM (short username oryx)
I20260806 04:35:04.064005 25600 webserver.cc:460] Webserver started at http://127.12.15.193:44987/ using document root <none> and password file <none>
I20260806 04:35:04.064589 25600 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:04.064664 25600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:04.068635 25600 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:35:04.070798 25647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.072132 25600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:35:04.072238 25600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "4af1363dc62b4fd1a10263de87715538"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "08a8c89724bd4557cb6d6f294f2b28cb"
server_key_iv: "ae43a04bea4c9a97e11a636b9652a4e3"
server_key_version: "encryptionkey@0"
I20260806 04:35:04.072809 25600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:04.099263 25600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:04.102298 25600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:04.102489 25600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:04.103106 25600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:04.103945 25600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:04.103992 25600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.104046 25600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:04.104076 25600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.112743 25600 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:35603
I20260806 04:35:04.112805 25832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:35603 every 8 connection(s)
I20260806 04:35:04.113756 25600 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:35:04.122740 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 25600
I20260806 04:35:04.123258 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:35323
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:34299
--webserver_port=40929
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.126739 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.115916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:42329)
0806 04:35:04.116417 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.116464 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.117632 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.118721 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.118729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.119166 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.119626 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.119640 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.120943 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.120946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.121796 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.121806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.122546 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.122551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.122725 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.123340 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.123362 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.125973 (+ 2611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
W20260806 04:35:04.127122 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.115792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:42329 (local address 127.12.15.254:34299)
0806 04:35:04.116361 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.116384 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.116569 (+ 185us) server_negotiation.cc:407] Connection header received
0806 04:35:04.118023 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.118034 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.118324 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.118657 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.119794 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.120815 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.121917 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.122376 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.122549 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.126279 (+ 3730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:42329: BlockingRecv error: recv got EOF from 127.12.15.193:42329 (error 108)
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":134,"threads_started":1}
W20260806 04:35:04.127218 25834 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:34299 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.15.254:34299: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.132891 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.127684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:59163)
0806 04:35:04.127783 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.127797 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.127868 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.128105 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.128106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.128233 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.128328 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.128332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.129361 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.129364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.129929 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.129938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.130484 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.130486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.130550 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.131002 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.131013 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.132795 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 04:35:04.133095 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.127770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:59163 (local address 127.12.15.254:34299)
0806 04:35:04.127873 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.127877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.127916 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:35:04.127954 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.127957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.128002 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.128084 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.128466 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.129243 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.130046 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.130371 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.130479 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.132966 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:59163: BlockingRecv error: recv got EOF from 127.12.15.193:59163 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.139271 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.133389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:49949)
0806 04:35:04.133726 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.133741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.133849 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.134081 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.134085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.134287 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.134404 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.134412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.135072 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.135076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.135966 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.135977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.136579 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.136583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.136692 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.137171 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.137184 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.139099 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
W20260806 04:35:04.139338 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.133468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49949 (local address 127.12.15.254:34299)
0806 04:35:04.133591 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.133595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.133814 (+ 219us) server_negotiation.cc:407] Connection header received
0806 04:35:04.133931 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.133934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.133974 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.134034 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.134503 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.134965 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.136167 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.136462 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.136554 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.139235 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:49949: BlockingRecv error: recv got EOF from 127.12.15.193:49949 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 04:35:04.139495 25834 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:35:04.240873 25838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:04.241111 25838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:04.241153 25838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:04.245078 25838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:04.245131 25838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:04.245193 25838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:35:04.250360 25838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.194.keytab
--principal=oryx/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:35323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=40929
--tserver_master_addrs=127.12.15.254:34299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:04.251348 25838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:04.252367 25838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:04.263504 25852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:04.263573 25855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:04.263723 25851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:04.263773 25838 server_base.cc:1034] running on GCE node
I20260806 04:35:04.264268 25838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:04.265043 25838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:04.266181 25838 hybrid_clock.cc:648] HybridClock initialized: now 1785990904266161 us; error 39 us; skew 500 ppm
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990904, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:04.269428 25838 init.cc:377] Logged in from keytab as oryx/127.12.15.194@KRBTEST.COM (short username oryx)
I20260806 04:35:04.270287 25838 webserver.cc:460] Webserver started at http://127.12.15.194:40929/ using document root <none> and password file <none>
I20260806 04:35:04.270963 25838 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:04.271068 25838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:04.275660 25838 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:35:04.278149 25865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.279834 25838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:35:04.280009 25838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "ce25e926c3694ff685bc0e518be52065"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "1fa4c34f801ba7d72de91a8055a88cb9"
server_key_iv: "cd673109cc74548edc810ad83d877f94"
server_key_version: "encryptionkey@0"
I20260806 04:35:04.280857 25838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:04.311762 25838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:04.314862 25838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:04.315160 25838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:04.315816 25838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:04.316725 25838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:04.316774 25838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.316815 25838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:04.316829 25838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.325729 25838 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:35323
I20260806 04:35:04.325831 26047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:35323 every 8 connection(s)
I20260806 04:35:04.327060 25838 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:35:04.335032 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 25838
I20260806 04:35:04.335472 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:39697
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:34299
--webserver_port=44631
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:35:04.338711 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.329392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:60427 (local address 127.12.15.254:34299)
0806 04:35:04.329531 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.329535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.330312 (+ 777us) server_negotiation.cc:407] Connection header received
0806 04:35:04.331625 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.331629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.331707 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.331783 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.333218 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.333769 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.334923 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.335162 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.335253 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.338573 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:60427: BlockingRecv error: recv got EOF from 127.12.15.194:60427 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:35:04.339355 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.329653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:60427)
0806 04:35:04.330155 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.330199 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.331404 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.331976 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.331984 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.332405 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.333050 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.333067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.333892 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.333894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.334797 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.334807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.335324 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.335329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.335487 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.336099 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.336116 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.338411 (+ 2295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":113,"threads_started":1}
W20260806 04:35:04.340217 26051 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:34299 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.15.254:34299: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.347030 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.340919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:37927)
0806 04:35:04.341108 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.341123 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.341221 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.341456 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.341459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.341646 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.341774 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.341782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.342571 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.342575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.343402 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.343415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.344089 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.344092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.344195 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.344725 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.344741 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.346873 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115}
W20260806 04:35:04.347225 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.341023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:37927 (local address 127.12.15.254:34299)
0806 04:35:04.341133 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.341136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.341170 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:35:04.341308 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.341310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.341348 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.341416 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.341916 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.342456 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.343621 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.343917 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.344064 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.347086 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:37927: BlockingRecv error: recv got EOF from 127.12.15.194:37927 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.353500 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.347928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:49239)
0806 04:35:04.348095 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.348108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.348180 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.348490 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.348493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.348665 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.348759 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.348765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.349512 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.349515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.350366 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.350376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.350834 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.350838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.350919 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.351358 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.351371 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.353360 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 04:35:04.353657 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.348043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:49239 (local address 127.12.15.254:34299)
0806 04:35:04.348149 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.348152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.348166 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:35:04.348281 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.348283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.348327 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.348473 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.348893 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.349421 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.350491 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.350734 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.350822 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.353550 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:49239: BlockingRecv error: recv got EOF from 127.12.15.194:49239 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 04:35:04.353848 26051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:35:04.454802 26055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:04.455030 26055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:04.455086 26055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:04.459465 26055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:04.459523 26055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:04.459607 26055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:35:04.464764 26055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/krb5kdc/oryx_127.12.15.195.keytab
--principal=oryx/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:39697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=44631
--tserver_master_addrs=127.12.15.254:34299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:04.465749 26055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:04.466727 26055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:04.476043 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:04.476050 26063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:04.476145 26062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:04.476365 26055 server_base.cc:1034] running on GCE node
I20260806 04:35:04.476792 26055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:04.477438 26055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:04.478621 26055 hybrid_clock.cc:648] HybridClock initialized: now 1785990904478589 us; error 43 us; skew 500 ppm
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990904, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:04.481344 26055 init.cc:377] Logged in from keytab as oryx/127.12.15.195@KRBTEST.COM (short username oryx)
I20260806 04:35:04.482254 26055 webserver.cc:460] Webserver started at http://127.12.15.195:44631/ using document root <none> and password file <none>
I20260806 04:35:04.482807 26055 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:04.482888 26055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:04.486730 26055 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:35:04.488833 26083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.490110 26055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:35:04.490218 26055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "84c8bfcf964d4798805404a06548e7d6"
format_stamp: "Formatted at 2026-08-06 04:35:03 on dist-test-slave-xt4k"
server_key: "0ee34a6e4bb38df027b06e012aa20d9b"
server_key_iv: "c4c07ae0b60b7fea7c2d9db5621e9887"
server_key_version: "encryptionkey@0"
I20260806 04:35:04.490777 26055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:04.510039 26055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:04.512908 26055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:04.513099 26055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:04.513703 26055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:04.514549 26055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:04.514597 26055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.514652 26055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:04.514680 26055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:04.523535 26055 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39697
I20260806 04:35:04.523561 26278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39697 every 8 connection(s)
I20260806 04:35:04.524580 26055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:35:04.533290 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 26055
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 04:35:04.536106 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.526364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:40467 (local address 127.12.15.254:34299)
0806 04:35:04.526485 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.526488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.527339 (+ 851us) server_negotiation.cc:407] Connection header received
0806 04:35:04.528848 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.528852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.528911 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.528993 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.530181 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.530698 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.531795 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.532004 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.532089 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.535967 (+ 3878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:40467: BlockingRecv error: recv got EOF from 127.12.15.195:40467 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:35:04.536733 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.526715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:40467)
0806 04:35:04.527188 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.527229 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.528544 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.529190 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.529197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.529555 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.530031 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.530044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.530811 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.530814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.531673 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.531683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.532146 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.532150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.532306 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.532888 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.532905 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.535841 (+ 2936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
W20260806 04:35:04.537215 26280 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:34299 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.15.254:34299: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.542426 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.537645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:37213)
0806 04:35:04.537758 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.537771 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.537834 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.538056 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.538058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.538182 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.538269 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.538274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.538934 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.538937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.539523 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.539533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.539930 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.539933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.540018 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.540477 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.540487 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.542287 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 04:35:04.542531 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.537738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:37213 (local address 127.12.15.254:34299)
0806 04:35:04.537836 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.537840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.537852 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:35:04.537908 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.537911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.537953 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.538034 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.538360 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.538829 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.539634 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.539835 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.539916 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.542428 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:37213: BlockingRecv error: recv got EOF from 127.12.15.195:37213 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 04:35:04.543500 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.534422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.12.15.254:34299)
0806 04:35:04.534668 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.534671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.534682 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:35:04.534711 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.534713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.534748 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.534826 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.535709 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.536406 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.537428 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.537722 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.537819 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.539186 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:04.539191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:04.539206 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:04.539232 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:04.541137 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:04.541629 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:04.541636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:04.541643 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:04.541803 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:04.542169 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:04.542172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:04.542173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:04.542608 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:04.542813 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:04.543102 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 04:35:04.543350 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:04 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:04.548166 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:04.542954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:52635)
0806 04:35:04.543264 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:04.543277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:04.543353 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:04.543570 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:04.543572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:04.543744 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:04.543826 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.543832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.544770 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.544773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.545526 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:04.545537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.545969 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.545972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:04.546047 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.546449 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:04.546463 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:04.548065 (+ 1602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":98,"threads_started":1}
W20260806 04:35:04.548324 25836 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:04.543035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:52635 (local address 127.12.15.254:34299)
0806 04:35:04.543162 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:04.543165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:04.543323 (+ 158us) server_negotiation.cc:407] Connection header received
0806 04:35:04.543430 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:04.543433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:04.543475 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:04.543559 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:04.543947 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.544663 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.545628 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:04.545870 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:04.545956 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:04.548221 (+ 2265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:52635: BlockingRecv error: recv got EOF from 127.12.15.195:52635 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 04:35:04.548403 26280 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 04:35:05 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:05 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:05.147320 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:05.140247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:38701)
0806 04:35:05.140542 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:05.140560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:05.140663 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:05.141227 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:05.141229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:05.141425 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:05.141606 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:05.141613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.142558 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.142561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:05.143190 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:05.143200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.143897 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.143900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:05.143986 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:05.144762 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:05.144774 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:05.147183 (+ 2409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":99,"threads_started":1}
W20260806 04:35:05.147490 26290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:05.140524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:38701 (local address 127.12.15.254:34299)
0806 04:35:05.140870 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:05.140874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:05.140894 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:35:05.140954 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:05.140957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:05.141016 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:05.141142 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:05.141770 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.142416 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.143370 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.143743 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.143857 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:05.147358 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:38701: BlockingRecv error: recv got EOF from 127.12.15.193:38701 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
Aug 06 04:35:05 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:05 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:05.360080 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:05.354464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:54011)
0806 04:35:05.354738 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:05.354754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:05.354854 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:05.355153 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:05.355156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:05.355338 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:05.355442 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:05.355448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.356317 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.356320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:05.356917 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:05.356926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.357346 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.357349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:05.357407 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:05.357914 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:05.357926 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:05.359833 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":79,"threads_started":1}
W20260806 04:35:05.360198 26290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:05.354576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:54011 (local address 127.12.15.254:34299)
0806 04:35:05.354781 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:05.354788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:05.354807 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:35:05.354932 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:05.354935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:05.354987 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:05.355077 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:05.355575 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.356208 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.357034 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.357253 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.357344 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:05.360052 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:54011: BlockingRecv error: recv got EOF from 127.12.15.194:54011 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 04:35:05 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:05 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:05.554567 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:05.549077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:37923)
0806 04:35:05.549341 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:05.549354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:05.549455 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:05.549662 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:05.549664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:05.549855 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:05.549966 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:05.549971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.550787 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.550790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:05.551378 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:05.551387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.551871 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.551875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:05.551948 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:05.552500 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:05.552512 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:05.554397 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":97,"threads_started":1}
W20260806 04:35:05.554639 26290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:05.549137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:37923 (local address 127.12.15.254:34299)
0806 04:35:05.549263 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:05.549268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:05.549417 (+ 149us) server_negotiation.cc:407] Connection header received
0806 04:35:05.549526 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:05.549528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:05.549573 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:05.549659 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:05.550116 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.550659 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.551495 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:05.551739 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:05.551824 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:05.554546 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:37923: BlockingRecv error: recv got EOF from 127.12.15.195:37923 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 04:35:06 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:06 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:06.153391 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:06.148196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:45395)
0806 04:35:06.148494 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:06.148511 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:06.148609 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:06.148829 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:06.148831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:06.149027 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:06.149137 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:06.149143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.149874 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.149877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:06.150484 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:06.150495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.150928 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.150931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:06.151005 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:06.151485 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:06.151498 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:06.153252 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":72,"threads_started":1}
W20260806 04:35:06.153535 26295 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:06.148277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:45395 (local address 127.12.15.254:34299)
0806 04:35:06.148567 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:06.148571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:06.148584 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:35:06.148683 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:06.148685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:06.148730 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:06.148805 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:06.149231 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.149754 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.150608 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.150831 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.150921 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:06.153394 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:45395: BlockingRecv error: recv got EOF from 127.12.15.193:45395 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":130,"threads_started":1}
Aug 06 04:35:06 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:06 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:06.366190 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:06.361052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:43525)
0806 04:35:06.361329 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:06.361342 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:06.361441 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:06.361633 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:06.361635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:06.361815 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:06.361909 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:06.361914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.362653 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.362657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:06.363208 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:06.363215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.363707 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.363709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:06.363772 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:06.364242 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:06.364253 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:06.365992 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
W20260806 04:35:06.366250 26295 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:06.361143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:43525 (local address 127.12.15.254:34299)
0806 04:35:06.361287 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:06.361291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:06.361390 (+ 99us) server_negotiation.cc:407] Connection header received
0806 04:35:06.361507 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:06.361509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:06.361552 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:06.361623 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:06.362017 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.362561 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.363301 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.363602 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.363691 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:06.366147 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:43525: BlockingRecv error: recv got EOF from 127.12.15.194:43525 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 04:35:06 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:06 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:06.560880 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:06.555348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:45831)
0806 04:35:06.555707 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:06.555723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:06.555846 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:06.556089 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:06.556092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:06.556294 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:06.556412 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:06.556418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.557209 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.557213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:06.557764 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:06.557773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.558195 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.558199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:06.558278 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:06.558823 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:06.558835 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:06.560723 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
W20260806 04:35:06.560968 26295 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:06.555524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:45831 (local address 127.12.15.254:34299)
0806 04:35:06.555707 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:06.555711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:06.555794 (+ 83us) server_negotiation.cc:407] Connection header received
0806 04:35:06.555933 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:06.555936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:06.555986 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:06.556051 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:06.556523 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.557097 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.557878 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:06.558093 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:06.558178 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:06.560865 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:45831: BlockingRecv error: recv got EOF from 127.12.15.195:45831 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 06 04:35:07 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:07 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:07.160154 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:07.154245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:43759)
0806 04:35:07.154546 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:07.154561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:07.154665 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:07.154980 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:07.154983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:07.155233 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:07.155368 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:07.155375 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.156474 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.156478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:07.157178 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:07.157190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.157617 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.157620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:07.157682 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:07.158179 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:07.158194 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:07.160019 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
W20260806 04:35:07.160315 26303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:07.154328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:43759 (local address 127.12.15.254:34299)
0806 04:35:07.154563 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:07.154566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:07.154612 (+ 46us) server_negotiation.cc:407] Connection header received
0806 04:35:07.154756 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:07.154759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:07.154804 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:07.154914 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:07.155522 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.156360 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.157286 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.157505 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.157591 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:07.160170 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:43759: BlockingRecv error: recv got EOF from 127.12.15.193:43759 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":74,"threads_started":1}
Aug 06 04:35:07 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:07 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:07.372308 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:07.367056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:48875)
0806 04:35:07.367372 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:07.367384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:07.367484 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:07.367698 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:07.367700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:07.367876 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:07.367970 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:07.367975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.368687 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.368690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:07.369236 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:07.369244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.369677 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.369680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:07.369806 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:07.370316 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:07.370328 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:07.372154 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":99,"threads_started":1}
W20260806 04:35:07.372442 26303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:07.367104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:48875 (local address 127.12.15.254:34299)
0806 04:35:07.367255 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:07.367259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:07.367473 (+ 214us) server_negotiation.cc:407] Connection header received
0806 04:35:07.367554 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:07.367556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:07.367604 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:07.367672 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:07.368059 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.368597 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.369355 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.369587 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.369665 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:07.372307 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:48875: BlockingRecv error: recv got EOF from 127.12.15.194:48875 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 04:35:07 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:07 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:07.571431 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:07.561762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:48353)
0806 04:35:07.562047 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:07.562060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:07.562166 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:07.562389 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:07.562391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:07.562615 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:07.562718 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:07.562724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.563573 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.563579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:07.564405 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:07.564421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.565135 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.565139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:07.565209 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:07.569330 (+ 4121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:07.569344 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:07.571308 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
W20260806 04:35:07.571604 26303 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:07.561910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:48353 (local address 127.12.15.254:34299)
0806 04:35:07.562041 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:07.562045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:07.562155 (+ 110us) server_negotiation.cc:407] Connection header received
0806 04:35:07.562236 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:07.562238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:07.562290 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:07.562363 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:07.562831 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.563351 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.564544 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:07.564943 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:07.565098 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:07.571487 (+ 6389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:48353: BlockingRecv error: recv got EOF from 127.12.15.195:48353 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 04:35:08 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:08 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:08.169636 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:08.161065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:52711)
0806 04:35:08.161342 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:08.161359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:08.161478 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:08.161779 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:08.161783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:08.161990 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:08.162091 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:08.162097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.162934 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.162938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:08.163538 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:08.163548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.164024 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.164029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:08.164096 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:08.167540 (+ 3444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:08.167554 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:08.169506 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
W20260806 04:35:08.169781 26311 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:08.161178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:52711 (local address 127.12.15.254:34299)
0806 04:35:08.161491 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:08.161494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:08.161512 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:35:08.161563 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:08.161566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:08.161617 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:08.161722 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:08.162204 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.162810 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.163667 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.163911 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.163999 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:08.169657 (+ 5658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:52711: BlockingRecv error: recv got EOF from 127.12.15.193:52711 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
Aug 06 04:35:08 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:08 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:08.378436 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:08.373213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:33009)
0806 04:35:08.373511 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:08.373525 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:08.373625 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:08.373876 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:08.373879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:08.374060 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:08.374166 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:08.374171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.374881 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.374884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:08.375446 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:08.375455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.375869 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.375872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:08.375948 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:08.376463 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:08.376474 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:08.378284 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":89,"threads_started":1}
W20260806 04:35:08.378590 26311 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:08.373287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33009 (local address 127.12.15.254:34299)
0806 04:35:08.373422 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:08.373426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:08.373584 (+ 158us) server_negotiation.cc:407] Connection header received
0806 04:35:08.373696 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:08.373699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:08.373743 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:08.373812 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:08.374262 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.374783 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.375531 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.375759 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.375826 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:08.378487 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:33009: BlockingRecv error: recv got EOF from 127.12.15.194:33009 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 04:35:08 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:08 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:08.577831 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:08.572256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:56753)
0806 04:35:08.572588 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:08.572601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:08.572790 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:08.573090 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:08.573093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:08.573297 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:08.573392 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:08.573398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.574121 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.574124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:08.574679 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:08.574687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.575093 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.575097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:08.575174 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:08.575666 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:08.575677 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:08.577713 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":167,"threads_started":1}
W20260806 04:35:08.577998 26311 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:08.572362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:56753 (local address 127.12.15.254:34299)
0806 04:35:08.572481 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:08.572486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:08.572671 (+ 185us) server_negotiation.cc:407] Connection header received
0806 04:35:08.572933 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:08.572936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:08.572988 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:08.573070 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:08.573499 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.574012 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.574784 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:08.574994 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:08.575088 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:08.577876 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:56753: BlockingRecv error: recv got EOF from 127.12.15.195:56753 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 04:35:09 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:09 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:09.175567 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:09.170420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:51899)
0806 04:35:09.170836 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:09.170851 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:09.170943 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:09.171163 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:09.171165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:09.171347 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:09.171460 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:09.171466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.172166 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.172169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:09.172750 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:09.172759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.173157 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.173159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:09.173218 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:09.173672 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:09.173683 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:09.175420 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":83,"threads_started":1}
W20260806 04:35:09.175737 26318 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:09.170485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:51899 (local address 127.12.15.254:34299)
0806 04:35:09.170757 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:09.170761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:09.170919 (+ 158us) server_negotiation.cc:407] Connection header received
0806 04:35:09.171008 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:09.171011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:09.171056 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:09.171133 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:09.171562 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.172077 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.172851 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.173071 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.173149 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:09.175584 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:51899: BlockingRecv error: recv got EOF from 127.12.15.193:51899 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":104,"threads_started":1}
Aug 06 04:35:09 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:09 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:09.385028 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:09.379472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:44411)
0806 04:35:09.379745 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:09.379761 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:09.379874 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:09.380162 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:09.380165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:09.380445 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:09.380570 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:09.380579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.381481 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.381484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:09.382039 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:09.382048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.382514 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.382517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:09.382596 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:09.383096 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:09.383108 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:09.384907 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
W20260806 04:35:09.385167 26318 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:09.379660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:44411 (local address 127.12.15.254:34299)
0806 04:35:09.379803 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:09.379806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:09.379820 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:35:09.379980 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:09.379983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:09.380034 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:09.380128 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:09.380697 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.381377 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.382152 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.382376 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.382468 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:09.385055 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:44411: BlockingRecv error: recv got EOF from 127.12.15.194:44411 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 04:35:09 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:09 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:09.583874 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:09.578615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:46627)
0806 04:35:09.578900 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:09.578914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:09.579020 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:09.579228 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:09.579230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:09.579416 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:09.579513 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:09.579518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.580267 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.580270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:09.580874 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:09.580883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.581326 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.581330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:09.581401 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:09.581925 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:09.581937 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:09.583733 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
W20260806 04:35:09.584067 26318 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:09.578694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:46627 (local address 127.12.15.254:34299)
0806 04:35:09.578815 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:09.578819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:09.578984 (+ 165us) server_negotiation.cc:407] Connection header received
0806 04:35:09.579090 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:09.579092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:09.579136 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:09.579205 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:09.579636 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.580165 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.580978 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:09.581197 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:09.581277 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:09.583943 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:46627: BlockingRecv error: recv got EOF from 127.12.15.195:46627 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 04:35:10 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:10 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:10.181962 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:10.176421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:36021)
0806 04:35:10.176700 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:10.176715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:10.176816 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:10.177042 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:10.177044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:10.177233 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:10.177334 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:10.177339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.178184 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.178188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:10.178844 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:10.178857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.179372 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.179375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:10.179471 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:10.179923 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:10.179935 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:10.181811 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
W20260806 04:35:10.182083 26323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:10.176559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:36021 (local address 127.12.15.254:34299)
0806 04:35:10.176772 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:10.176775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:10.176790 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:35:10.176895 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:10.176897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:10.176941 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:10.177028 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:10.177423 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.178076 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.178959 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.179234 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.179315 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:10.181954 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:36021: BlockingRecv error: recv got EOF from 127.12.15.193:36021 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":61,"threads_started":1}
Aug 06 04:35:10 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:10 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:10.391553 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:10.385944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:46227)
0806 04:35:10.386196 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:10.386209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:10.386317 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:10.386614 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:10.386617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:10.386834 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:10.386935 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:10.386941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.387770 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.387773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:10.388415 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:10.388426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.388899 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.388902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:10.388987 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:10.389510 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:10.389525 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:10.391435 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
W20260806 04:35:10.391680 26323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:10.386025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:46227 (local address 127.12.15.254:34299)
0806 04:35:10.386204 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:10.386208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:10.386277 (+ 69us) server_negotiation.cc:407] Connection header received
0806 04:35:10.386437 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:10.386440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:10.386494 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:10.386568 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:10.387085 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.387644 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.388542 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.388776 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.388864 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:10.391564 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:46227: BlockingRecv error: recv got EOF from 127.12.15.194:46227 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 04:35:10 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:10 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:10.590121 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:10.584680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:48139)
0806 04:35:10.584971 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:10.584984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:10.585090 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:10.585339 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:10.585341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:10.585518 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:10.585618 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:10.585623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.586460 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.586464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:10.587220 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:10.587232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.587656 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.587658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:10.587717 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:10.588196 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:10.588210 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:10.589987 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":112,"threads_started":1}
W20260806 04:35:10.590222 26323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:10.584781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:48139 (local address 127.12.15.254:34299)
0806 04:35:10.584946 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:10.584951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:10.585055 (+ 104us) server_negotiation.cc:407] Connection header received
0806 04:35:10.585183 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:10.585186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:10.585244 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:10.585325 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:10.585712 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.586317 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.587326 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:10.587560 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:10.587638 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:10.590113 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:48139: BlockingRecv error: recv got EOF from 127.12.15.195:48139 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 04:35:11 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:11 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:11.188077 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:11.182795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:43793)
0806 04:35:11.183100 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:11.183113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:11.183207 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:11.183446 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:11.183448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:11.183629 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:11.183728 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:11.183733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.184557 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.184560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:11.185103 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:11.185112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.185521 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.185524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:11.185604 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:11.186116 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:11.186131 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:11.187958 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":78,"threads_started":1}
W20260806 04:35:11.188234 26335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:11.182894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:43793 (local address 127.12.15.254:34299)
0806 04:35:11.183140 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:11.183144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:11.183160 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:35:11.183288 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:11.183290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:11.183333 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:11.183422 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:11.183825 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.184440 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.185209 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.185421 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.185506 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:11.188090 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:43793: BlockingRecv error: recv got EOF from 127.12.15.193:43793 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
Aug 06 04:35:11 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:11 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:11.397601 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:11.392224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:58217)
0806 04:35:11.392575 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:11.392591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:11.392690 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:11.392903 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:11.392906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:11.393086 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:11.393180 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:11.393185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.393865 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.393868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:11.394438 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:11.394447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.394910 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.394913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:11.394974 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:11.395619 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:11.395631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:11.397462 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":105,"threads_started":1}
W20260806 04:35:11.397724 26335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:11.392282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:58217 (local address 127.12.15.254:34299)
0806 04:35:11.392454 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:11.392458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:11.392643 (+ 185us) server_negotiation.cc:407] Connection header received
0806 04:35:11.392766 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:11.392768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:11.392812 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:11.392873 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:11.393257 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.393769 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.394563 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.394799 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.394888 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:11.397609 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:58217: BlockingRecv error: recv got EOF from 127.12.15.194:58217 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 04:35:11 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:11 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:11.596015 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:11.590863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:58809)
0806 04:35:11.591137 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:11.591149 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:11.591261 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:11.591522 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:11.591525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:11.591720 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:11.591815 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:11.591821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.592575 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.592579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:11.593122 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:11.593131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.593546 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.593550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:11.593630 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:11.594127 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:11.594138 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:11.595864 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":68,"threads_started":1}
W20260806 04:35:11.596110 26335 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:11.590938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:58809 (local address 127.12.15.254:34299)
0806 04:35:11.591072 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:11.591076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:11.591215 (+ 139us) server_negotiation.cc:407] Connection header received
0806 04:35:11.591346 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:11.591350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:11.591395 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:11.591470 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:11.591926 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.592480 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.593231 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:11.593441 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:11.593527 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:11.595996 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:58809: BlockingRecv error: recv got EOF from 127.12.15.195:58809 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 04:35:12 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:12 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:12.193809 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:12.188754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:49491)
0806 04:35:12.188998 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:12.189010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:12.189094 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:12.189341 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:12.189342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:12.189516 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:12.189633 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:12.189639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.190516 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.190519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:12.191084 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:12.191093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.191466 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.191468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:12.191523 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:12.191950 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:12.191960 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:12.193663 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
W20260806 04:35:12.193934 26342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:12.188898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49491 (local address 127.12.15.254:34299)
0806 04:35:12.189127 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:12.189131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:12.189143 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:35:12.189180 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:12.189183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:12.189242 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:12.189335 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:12.189771 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.190393 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.191178 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.191390 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.191481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:12.193798 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:49491: BlockingRecv error: recv got EOF from 127.12.15.193:49491 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
Aug 06 04:35:12 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:12 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:12.403832 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:12.398373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:58693)
0806 04:35:12.398694 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:12.398712 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:12.398819 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:12.399043 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:12.399044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:12.399223 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:12.399317 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:12.399322 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.400216 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.400219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:12.400809 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:12.400818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.401255 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.401257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:12.401313 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:12.401832 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:12.401845 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:12.403682 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":144,"threads_started":1}
W20260806 04:35:12.403924 26342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:12.398446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:58693 (local address 127.12.15.254:34299)
0806 04:35:12.398597 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:12.398602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:12.398809 (+ 207us) server_negotiation.cc:407] Connection header received
0806 04:35:12.398887 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:12.398890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:12.398944 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:12.399013 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:12.399430 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.400115 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.400923 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.401166 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.401251 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:12.403819 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:58693: BlockingRecv error: recv got EOF from 127.12.15.194:58693 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 04:35:12 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:12 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:12.601804 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:12.596773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:36487)
0806 04:35:12.597041 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:12.597054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:12.597157 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:12.597362 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:12.597363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:12.597538 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:12.597634 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:12.597639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.598304 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.598306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:12.598859 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:12.598867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.599209 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.599211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:12.599269 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:12.599713 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:12.599723 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:12.601629 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
W20260806 04:35:12.601873 26342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:12.596845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:36487 (local address 127.12.15.254:34299)
0806 04:35:12.596990 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:12.596995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:12.597110 (+ 115us) server_negotiation.cc:407] Connection header received
0806 04:35:12.597228 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:12.597230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:12.597275 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:12.597339 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:12.597716 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.598216 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.598939 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:12.599135 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:12.599202 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:12.601773 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:36487: BlockingRecv error: recv got EOF from 127.12.15.195:36487 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 04:35:13 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:13 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:13.212841 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:13.194529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:37977)
0806 04:35:13.194795 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:13.194809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:13.194902 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:13.195240 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:13.195243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:13.195469 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:13.195577 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:13.195584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.196386 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.196390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:13.197038 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:13.197049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.197487 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.197491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:13.197584 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:13.210689 (+ 13105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:13.210706 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:13.212708 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
W20260806 04:35:13.213064 26348 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:13.194595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:37977 (local address 127.12.15.254:34299)
0806 04:35:13.194867 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:13.194870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:13.194884 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:35:13.195037 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:13.195040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:13.195088 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:13.195166 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:13.195705 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.196246 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.197144 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.197361 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.197439 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:13.212908 (+ 15469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:37977: BlockingRecv error: recv got EOF from 127.12.15.193:37977 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
Aug 06 04:35:13 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:13 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:13.409935 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:13.404609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:59407)
0806 04:35:13.404937 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:13.404951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:13.405066 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:13.405318 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:13.405321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:13.405511 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:13.405609 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:13.405615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.406345 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.406348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:13.406907 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:13.406916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.407364 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.407367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:13.407431 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:13.407934 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:13.407946 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:13.409792 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":75,"threads_started":1}
W20260806 04:35:13.410080 26348 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:13.404700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:59407 (local address 127.12.15.254:34299)
0806 04:35:13.404910 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:13.404913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:13.405016 (+ 103us) server_negotiation.cc:407] Connection header received
0806 04:35:13.405138 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:13.405140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:13.405183 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:13.405248 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:13.405724 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.406248 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.407027 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.407251 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.407338 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:13.409955 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:59407: BlockingRecv error: recv got EOF from 127.12.15.194:59407 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 06 04:35:13 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:13 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:13.607661 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:13.602601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:55487)
0806 04:35:13.602882 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:13.602897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:13.602998 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:13.603220 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:13.603222 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:13.603403 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:13.603500 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:13.603506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.604237 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.604239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:13.604796 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:13.604803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.605208 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.605210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:13.605271 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:13.605752 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:13.605763 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:13.607543 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":102,"threads_started":1}
W20260806 04:35:13.607807 26348 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:13.602601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:55487 (local address 127.12.15.254:34299)
0806 04:35:13.602732 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:13.602736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:13.602966 (+ 230us) server_negotiation.cc:407] Connection header received
0806 04:35:13.603078 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:13.603080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:13.603129 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:13.603199 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:13.603618 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.604158 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.604896 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:13.605118 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:13.605195 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:13.607681 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:55487: BlockingRecv error: recv got EOF from 127.12.15.195:55487 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 04:35:14 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:14 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:14.218755 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:14.213687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:49547)
0806 04:35:14.213979 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:14.213992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:14.214079 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:14.214317 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:14.214320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:14.214520 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:14.214632 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:14.214640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.215331 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.215333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:14.215885 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:14.215892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.216291 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.216293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:14.216385 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:14.216857 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:14.216869 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:14.218631 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
W20260806 04:35:14.218938 26356 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:14.213779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49547 (local address 127.12.15.254:34299)
0806 04:35:14.214024 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:14.214028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:14.214040 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:35:14.214156 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:14.214159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:14.214201 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:14.214274 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:14.214728 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.215250 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.215983 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.216206 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.216285 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:14.218795 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:49547: BlockingRecv error: recv got EOF from 127.12.15.193:49547 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
Aug 06 04:35:14 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:14 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:14.416013 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:14.410806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:54043)
0806 04:35:14.411091 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:14.411105 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:14.411216 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:14.411467 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:14.411470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:14.411677 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:14.411776 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:14.411783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.412545 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.412548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:14.413106 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:14.413115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.413525 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.413529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:14.413607 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:14.414085 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:14.414096 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:14.415851 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
W20260806 04:35:14.416093 26356 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:14.410845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:54043 (local address 127.12.15.254:34299)
0806 04:35:14.410985 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:14.410990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:14.411197 (+ 207us) server_negotiation.cc:407] Connection header received
0806 04:35:14.411318 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:14.411320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:14.411366 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:14.411432 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:14.411891 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.412448 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.413216 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.413426 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.413517 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:14.415990 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:54043: BlockingRecv error: recv got EOF from 127.12.15.194:54043 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 04:35:14 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:14 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:14.613814 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:14.608439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:56459)
0806 04:35:14.608761 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:14.608774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:14.608868 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:14.609080 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:14.609082 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:14.609259 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:14.609355 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:14.609361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.610058 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.610061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:14.610707 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:14.610721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.611187 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.611190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:14.611257 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:14.611786 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:14.611798 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:14.613664 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":112,"threads_started":1}
W20260806 04:35:14.613929 26356 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:14.608560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:56459 (local address 127.12.15.254:34299)
0806 04:35:14.608741 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:14.608744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:14.608835 (+ 91us) server_negotiation.cc:407] Connection header received
0806 04:35:14.608938 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:14.608940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:14.608984 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:14.609048 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:14.609437 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.609965 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.610846 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:14.611076 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:14.611168 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:14.613822 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:56459: BlockingRecv error: recv got EOF from 127.12.15.195:56459 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 04:35:15 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:15 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:15.225147 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:15.219571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:37183)
0806 04:35:15.219898 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:15.219911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:15.220011 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:15.220250 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:15.220251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:15.220456 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:15.220567 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:15.220573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.221479 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.221483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:15.222027 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:15.222036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.222573 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.222578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:15.222667 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:15.223211 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:15.223223 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:15.225031 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
W20260806 04:35:15.225287 26365 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:15.219677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:37183 (local address 127.12.15.254:34299)
0806 04:35:15.219951 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:15.219955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:15.219970 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:35:15.220099 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:15.220102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:15.220147 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:15.220223 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:15.220697 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.221371 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.222142 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.222423 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.222528 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:15.225158 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:37183: BlockingRecv error: recv got EOF from 127.12.15.193:37183 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
Aug 06 04:35:15 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:15 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:15.422446 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:15.416889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:60893)
0806 04:35:15.417334 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:15.417347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:15.417453 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:15.417759 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:15.417762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:15.418007 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:15.418133 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:15.418141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.418929 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.418932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:15.419493 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:15.419502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.419912 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.419915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:15.419986 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:15.420534 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:15.420546 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:15.422317 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":87,"threads_started":1}
W20260806 04:35:15.422559 26365 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:15.416966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:60893 (local address 127.12.15.254:34299)
0806 04:35:15.417115 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:15.417119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:15.417423 (+ 304us) server_negotiation.cc:407] Connection header received
0806 04:35:15.417543 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:15.417546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:15.417597 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:15.417696 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:15.418255 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.418828 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.419583 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.419795 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.419866 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:15.422462 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:60893: BlockingRecv error: recv got EOF from 127.12.15.194:60893 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 04:35:15 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:15 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:15.620311 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:15.614577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:54955)
0806 04:35:15.614850 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:15.614865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:15.614964 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:15.615182 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:15.615184 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:15.615359 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:15.615455 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:15.615460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.616358 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.616363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:15.617075 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:15.617086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.617516 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.617519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:15.617574 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:15.618104 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:15.618115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:15.620192 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":116,"threads_started":1}
W20260806 04:35:15.620484 26365 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:15.614631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:54955 (local address 127.12.15.254:34299)
0806 04:35:15.614772 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:15.614776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:15.614938 (+ 162us) server_negotiation.cc:407] Connection header received
0806 04:35:15.615043 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:15.615045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:15.615090 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:15.615159 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:15.615551 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.616225 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.617205 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:15.617426 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:15.617513 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:15.620362 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:54955: BlockingRecv error: recv got EOF from 127.12.15.195:54955 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 04:35:16 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:16 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:16.231001 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:16.225906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:58107)
0806 04:35:16.226244 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:16.226257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:16.226356 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:16.226579 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:16.226582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:16.226759 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:16.226857 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:16.226863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.227614 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.227616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:16.228153 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:16.228160 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.228588 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.228590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:16.228649 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:16.229121 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:16.229131 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:16.230862 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":145,"threads_started":1}
W20260806 04:35:16.231156 26371 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:16.225982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:58107 (local address 127.12.15.254:34299)
0806 04:35:16.226218 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:16.226221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:16.226317 (+ 96us) server_negotiation.cc:407] Connection header received
0806 04:35:16.226427 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:16.226430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:16.226472 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:16.226551 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:16.226980 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.227527 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.228242 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.228504 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.228589 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:16.231015 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:58107: BlockingRecv error: recv got EOF from 127.12.15.193:58107 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
Aug 06 04:35:16 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:16 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:16.428745 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:16.423208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:51083)
0806 04:35:16.423471 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:16.423484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:16.423603 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:16.423858 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:16.423861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:16.424048 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:16.424155 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:16.424161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.425027 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.425030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:16.425603 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:16.425612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.426008 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.426010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:16.426069 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:16.426517 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:16.426529 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:16.428585 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
W20260806 04:35:16.428861 26371 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:16.423293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:51083 (local address 127.12.15.254:34299)
0806 04:35:16.423472 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:16.423476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:16.423548 (+ 72us) server_negotiation.cc:407] Connection header received
0806 04:35:16.423676 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:16.423678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:16.423723 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:16.423788 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:16.424277 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.424919 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.425713 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.425925 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.426000 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:16.428759 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:51083: BlockingRecv error: recv got EOF from 127.12.15.194:51083 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 06 04:35:16 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:16 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:16.626304 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:16.621023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:48199)
0806 04:35:16.621299 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:16.621312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:16.621413 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:16.621634 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:16.621636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:16.621813 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:16.621908 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:16.621914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.622704 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.622707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:16.623268 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:16.623276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.623781 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.623784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:16.623856 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:16.624373 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:16.624385 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:16.626185 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":103,"threads_started":1}
W20260806 04:35:16.626471 26371 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:16.621101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:48199 (local address 127.12.15.254:34299)
0806 04:35:16.621234 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:16.621238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:16.621412 (+ 174us) server_negotiation.cc:407] Connection header received
0806 04:35:16.621481 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:16.621483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:16.621536 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:16.621610 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:16.622005 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.622565 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.623420 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:16.623649 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:16.623736 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:16.626342 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:48199: BlockingRecv error: recv got EOF from 127.12.15.195:48199 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 04:35:17 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:17 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:17.237154 26375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:17.231823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:33079)
0806 04:35:17.232120 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:17.232135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:17.232236 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:17.232483 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:17.232485 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:17.232674 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:17.232777 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:17.232783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.233577 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.233580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:17.234154 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:17.234163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.234611 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.234613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:17.234669 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:17.235140 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:17.235151 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:17.237011 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
W20260806 04:35:17.237267 26381 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:17.231939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:33079 (local address 127.12.15.254:34299)
0806 04:35:17.232187 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:17.232191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:17.232205 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:35:17.232306 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:17.232308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:17.232369 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:17.232457 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:17.232906 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.233472 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.234284 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.234515 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.234596 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:17.237151 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:33079: BlockingRecv error: recv got EOF from 127.12.15.193:33079 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
Aug 06 04:35:17 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:17 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:17.434755 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:17.429578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:33961)
0806 04:35:17.429874 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:17.429889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:17.429991 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:17.430205 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:17.430208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:17.430398 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:17.430503 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:17.430508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.431236 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.431239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:17.431852 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:17.431862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.432291 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.432294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:17.432391 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:17.432918 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:17.432933 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:17.434628 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
W20260806 04:35:17.434891 26381 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:17.429756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:33961 (local address 127.12.15.254:34299)
0806 04:35:17.429898 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:17.429902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:17.429951 (+ 49us) server_negotiation.cc:407] Connection header received
0806 04:35:17.430067 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:17.430069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:17.430114 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:17.430185 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:17.430592 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.431114 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.431960 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.432185 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.432261 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:17.434771 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:33961: BlockingRecv error: recv got EOF from 127.12.15.194:33961 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 04:35:17 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:17 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:17.632117 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:17.627119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:43141)
0806 04:35:17.627412 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:17.627426 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:17.627528 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:17.627733 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:17.627735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:17.627924 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:17.628019 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:17.628024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.628776 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.628778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:17.629352 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:17.629359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.629724 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.629726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:17.629782 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:17.630286 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:17.630298 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:17.631981 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":91,"threads_started":1}
W20260806 04:35:17.632221 26381 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:17.627166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:43141 (local address 127.12.15.254:34299)
0806 04:35:17.627304 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:17.627310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:17.627482 (+ 172us) server_negotiation.cc:407] Connection header received
0806 04:35:17.627593 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:17.627595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:17.627642 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:17.627706 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:17.628103 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.628682 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.629432 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:17.629647 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:17.629715 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:17.632117 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:43141: BlockingRecv error: recv got EOF from 127.12.15.195:43141 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 04:35:18 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:18 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:18.246354 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:18.237963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:49545)
0806 04:35:18.238244 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:18.238258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:18.238350 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:18.238577 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:18.238579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:18.238774 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:18.238870 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:18.238875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.239595 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.239597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:18.240211 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:18.240223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.240746 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.240750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:18.240838 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:18.244280 (+ 3442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:18.244293 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:18.246202 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
W20260806 04:35:18.246459 26385 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:18.238065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49545 (local address 127.12.15.254:34299)
0806 04:35:18.238345 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:18.238349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:18.238362 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:35:18.238435 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:18.238437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:18.238479 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:18.238551 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:18.238962 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.239519 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.240329 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.240635 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.240740 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:18.246344 (+ 5604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:49545: BlockingRecv error: recv got EOF from 127.12.15.193:49545 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":65,"threads_started":1}
Aug 06 04:35:18 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:18 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:18.440887 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:18.435592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:38653)
0806 04:35:18.435875 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:18.435891 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:18.436011 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:18.436298 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:18.436301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:18.436530 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:18.436627 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:18.436633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.437371 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.437375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:18.437942 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:18.437950 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.438375 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.438379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:18.438461 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:18.438961 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:18.438973 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:18.440727 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
W20260806 04:35:18.441090 26385 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:18.435683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:38653 (local address 127.12.15.254:34299)
0806 04:35:18.435829 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:18.435833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:18.435960 (+ 127us) server_negotiation.cc:407] Connection header received
0806 04:35:18.436119 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:18.436123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:18.436180 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:18.436251 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:18.436746 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.437269 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.438060 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.438275 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.438360 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:18.440872 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:38653: BlockingRecv error: recv got EOF from 127.12.15.194:38653 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 04:35:18 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:18 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.195@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:18.637887 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:18.632981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.195:43379)
0806 04:35:18.633219 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:18.633232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:18.633329 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:18.633530 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:18.633532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:18.633719 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:18.633814 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:18.633819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.634478 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.634480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:18.635031 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:18.635038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.635439 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.635442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:18.635512 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:18.636017 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:18.636029 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:18.637772 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
W20260806 04:35:18.638041 26385 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:18.633038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:43379 (local address 127.12.15.254:34299)
0806 04:35:18.633178 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:18.633181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:18.633290 (+ 109us) server_negotiation.cc:407] Connection header received
0806 04:35:18.633397 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:18.633399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:18.633445 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:18.633510 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:18.633892 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.634400 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.635111 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:18.635313 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:18.635383 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:18.637917 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.195:43379: BlockingRecv error: recv got EOF from 127.12.15.195:43379 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 04:35:19 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:19 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.193@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:19.252646 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:19.247210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.193:35557)
0806 04:35:19.247472 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:19.247488 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:19.247611 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:19.247885 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:19.247888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:19.248098 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:19.248212 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:19.248220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.249142 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.249145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:19.249725 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:19.249733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.250127 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.250129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:19.250187 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:19.250681 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:19.250693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:19.252491 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
W20260806 04:35:19.252755 26389 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:19.247351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:35557 (local address 127.12.15.254:34299)
0806 04:35:19.247606 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:19.247611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:19.247628 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:35:19.247693 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:19.247696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:19.247751 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:19.247837 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:19.248317 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.249031 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.249824 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.250044 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.250119 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:19.252640 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.193:35557: BlockingRecv error: recv got EOF from 127.12.15.193:35557 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
Aug 06 04:35:19 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 04:35:19 dist-test-slave-xt4k krb5kdc[24658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.194@KRBTEST.COM for oryx/127.12.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 04:35:19.447052 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:19.441774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:34299 (local address 127.12.15.194:40803)
0806 04:35:19.442024 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:19.442044 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:19.442135 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:19.442342 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:19.442344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:19.442530 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:19.442624 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:19.442630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.443307 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.443309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:19.443866 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:19.443872 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.444319 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.444321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:19.444445 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:19.444992 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:19.445008 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:19.446910 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.254:34299: Server oryx/127.12.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
W20260806 04:35:19.447145 26389 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:19.441885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:40803 (local address 127.12.15.254:34299)
0806 04:35:19.442031 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:19.442034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:19.442102 (+ 68us) server_negotiation.cc:407] Connection header received
0806 04:35:19.442206 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:19.442208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:19.442251 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:19.442320 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:19.442709 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.443230 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.443957 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.444183 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.444261 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:19.447039 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.194:40803: BlockingRecv error: recv got EOF from 127.12.15.194:40803 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:35:19.536765 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 25600
I20260806 04:35:19.558477 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 25838
I20260806 04:35:19.569114 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 26055
I20260806 04:35:19.578420 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 25491
2026-08-06T04:35:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18192 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T04:35:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:35:19Z Disabled control of system clock
I20260806 04:35:19.616978 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36957
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 04:35:19.731194 26401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:19.731469 26401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:19.731503 26401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:19.735512 26401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:35:19.735579 26401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:19.735594 26401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:19.735610 26401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:35:19.735623 26401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:35:19.740373 26401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36957
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:19.741458 26401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:19.742867 26401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:19.751307 26413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:19.751297 26417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:19.751461 26401 server_base.cc:1034] running on GCE node
W20260806 04:35:19.751621 26415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:19.752164 26401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:19.753225 26401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:19.754396 26401 hybrid_clock.cc:648] HybridClock initialized: now 1785990919754373 us; error 39 us; skew 500 ppm
I20260806 04:35:19.756181 26401 webserver.cc:460] Webserver started at http://127.12.15.254:35699/ using document root <none> and password file <none>
I20260806 04:35:19.756773 26401 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:19.756843 26401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:19.757050 26401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:19.759106 26401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "eb4d13886ee94577b5750e3cba81fd26"
format_stamp: "Formatted at 2026-08-06 04:35:19 on dist-test-slave-xt4k"
server_key: "315f4faa7347e0821f575cdeaf315f95"
server_key_iv: "fedec2805834b8f811472a1c49882915"
server_key_version: "encryptionkey@0"
I20260806 04:35:19.759632 26401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "eb4d13886ee94577b5750e3cba81fd26"
format_stamp: "Formatted at 2026-08-06 04:35:19 on dist-test-slave-xt4k"
server_key: "315f4faa7347e0821f575cdeaf315f95"
server_key_iv: "fedec2805834b8f811472a1c49882915"
server_key_version: "encryptionkey@0"
I20260806 04:35:19.763314 26401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:35:19.765858 26425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:19.767140 26401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:35:19.767270 26401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "eb4d13886ee94577b5750e3cba81fd26"
format_stamp: "Formatted at 2026-08-06 04:35:19 on dist-test-slave-xt4k"
server_key: "315f4faa7347e0821f575cdeaf315f95"
server_key_iv: "fedec2805834b8f811472a1c49882915"
server_key_version: "encryptionkey@0"
I20260806 04:35:19.767376 26401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:19.818439 26401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:19.819183 26401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:19.819362 26401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:19.826241 26401 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36957
I20260806 04:35:19.826246 26508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36957 every 8 connection(s)
I20260806 04:35:19.827173 26401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:35:19.830350 26513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:19.833637 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 26401
I20260806 04:35:19.833737 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:35:19.834004 12351 external_mini_cluster.cc:1428] Setting key 1b756580596dcaa8357d76f4851b75bf
I20260806 04:35:19.836087 26513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26: Bootstrap starting.
I20260806 04:35:19.838351 26513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:19.839174 26513 log.cc:824] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:19.841137 26513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26: No bootstrap required, opened a new log
I20260806 04:35:19.842916 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:19.835382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.12.15.254:36957)
0806 04:35:19.835972 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:19.835988 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:19.836021 (+ 33us) server_negotiation.cc:407] Connection header received
0806 04:35:19.836671 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:19.836689 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:19.837064 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:19.837321 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:19.837663 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.838634 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.839758 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:19.840217 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:19.840388 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:19.841266 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:19.841273 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:19.841290 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:19.841662 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:19.841721 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:19.841744 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:19.841835 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:19.841989 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 04:35:19.842228 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260806 04:35:19.844835 26513 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4d13886ee94577b5750e3cba81fd26" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36957 } }
I20260806 04:35:19.845099 26513 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:19.845196 26513 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4d13886ee94577b5750e3cba81fd26, State: Initialized, Role: FOLLOWER
I20260806 04:35:19.845579 26513 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4d13886ee94577b5750e3cba81fd26" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36957 } }
I20260806 04:35:19.845695 26513 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:35:19.845738 26513 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:35:19.845805 26513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:19.846598 26513 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4d13886ee94577b5750e3cba81fd26" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36957 } }
I20260806 04:35:19.846905 26513 leader_election.cc:302] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb4d13886ee94577b5750e3cba81fd26; no voters:
I20260806 04:35:19.847172 26513 leader_election.cc:288] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:35:19.847265 26519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:35:19.847481 26519 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [term 1 LEADER]: Becoming Leader. State: Replica: eb4d13886ee94577b5750e3cba81fd26, State: Running, Role: LEADER
I20260806 04:35:19.847849 26519 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4d13886ee94577b5750e3cba81fd26" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36957 } }
I20260806 04:35:19.848326 26513 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:35:19.849499 26523 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb4d13886ee94577b5750e3cba81fd26. Latest consensus state: current_term: 1 leader_uuid: "eb4d13886ee94577b5750e3cba81fd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4d13886ee94577b5750e3cba81fd26" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36957 } } }
I20260806 04:35:19.849640 26523 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:19.850020 26533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:35:19.849987 26522 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb4d13886ee94577b5750e3cba81fd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4d13886ee94577b5750e3cba81fd26" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36957 } } }
I20260806 04:35:19.850497 26522 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:19.852756 26533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:35:19.858186 26533 catalog_manager.cc:1347] Generated new cluster ID: c3739a7f0fab4bea99194600103e1a5c
I20260806 04:35:19.858247 26533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:35:19.866024 26533 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:35:19.867202 26533 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:35:19.873193 26533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb4d13886ee94577b5750e3cba81fd26: Generated new TSK 0
I20260806 04:35:19.874238 26533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:35:19.882807 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36957
--builtin_ntp_servers=127.12.15.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 04:35:19.998557 26550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:19.998821 26550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:19.998857 26550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:20.003101 26550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:20.003167 26550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:20.003237 26550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:35:20.008044 26550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:20.009195 26550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:20.010624 26550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:20.020030 26560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.020259 26550 server_base.cc:1034] running on GCE node
W20260806 04:35:20.020066 26558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.020107 26556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.020843 26550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:20.021476 26550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:20.022658 26550 hybrid_clock.cc:648] HybridClock initialized: now 1785990920022640 us; error 27 us; skew 500 ppm
I20260806 04:35:20.024513 26550 webserver.cc:460] Webserver started at http://127.12.15.193:40435/ using document root <none> and password file <none>
I20260806 04:35:20.025090 26550 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:20.025151 26550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:20.025341 26550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:20.027470 26550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "8f86f1ebbbe84a5cbe2610c4904acb1e"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "7730ed27578eac03c257a0bf4e071505"
server_key_iv: "f66034f698a32b46834a1d717d25f08d"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.028024 26550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "8f86f1ebbbe84a5cbe2610c4904acb1e"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "7730ed27578eac03c257a0bf4e071505"
server_key_iv: "f66034f698a32b46834a1d717d25f08d"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.031841 26550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:35:20.034420 26572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.035624 26550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:35:20.035733 26550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "8f86f1ebbbe84a5cbe2610c4904acb1e"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "7730ed27578eac03c257a0bf4e071505"
server_key_iv: "f66034f698a32b46834a1d717d25f08d"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.035828 26550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:20.048297 26550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:20.048965 26550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:20.049129 26550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:20.049678 26550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:20.050581 26550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:20.050630 26550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.050680 26550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:20.050711 26550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.058949 26550 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:36341
I20260806 04:35:20.058975 26753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:36341 every 8 connection(s)
I20260806 04:35:20.059792 26550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:35:20.062806 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 26550
I20260806 04:35:20.062942 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:35:20.063278 12351 external_mini_cluster.cc:1428] Setting key 5d1ac70d7da48629e87d8a95642d3f2f
I20260806 04:35:20.065351 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36957
--builtin_ntp_servers=127.12.15.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 04:35:20.069388 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.061622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:56669 (local address 127.12.15.254:36957)
0806 04:35:20.061742 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:20.061745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:20.062727 (+ 982us) server_negotiation.cc:407] Connection header received
0806 04:35:20.064404 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:20.064408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:20.064462 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:20.064540 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:20.065624 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.066146 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.067542 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.067869 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.067973 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:20.068959 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:20.068963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:20.068966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:20.068980 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:20.069002 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:20.069011 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:20.069082 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:20.069186 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 04:35:20.069241 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 04:35:20.070168 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.061889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36957 (local address 127.12.15.193:56669)
0806 04:35:20.062559 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:20.062612 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:20.064188 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:20.064716 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:20.064724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:20.064827 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:20.065484 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:20.065507 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.066265 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.066268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:20.067400 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:20.067411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.068000 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.068002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:20.068143 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:20.068717 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:35:20.068720 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:20.068759 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:35:20.068820 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:20.069081 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:20.069085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:20.069089 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:35:20.069173 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:35:20.069382 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":105,"threads_started":1}
I20260806 04:35:20.071462 26754 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36957
I20260806 04:35:20.071676 26754 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:20.072207 26754 heartbeater.cc:511] Master 127.12.15.254:36957 requested a full tablet report, sending...
I20260806 04:35:20.073779 26453 ts_manager.cc:194] Registered new tserver with Master: 8f86f1ebbbe84a5cbe2610c4904acb1e (127.12.15.193:36341)
I20260806 04:35:20.074894 26453 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.193:56669
W20260806 04:35:20.178562 26760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:20.178818 26760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:20.178854 26760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:20.182906 26760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:20.182960 26760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:20.183019 26760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:35:20.187674 26760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:20.188846 26760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:20.190258 26760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:20.199888 26768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.200009 26760 server_base.cc:1034] running on GCE node
W20260806 04:35:20.199855 26769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.199856 26773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.200664 26760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:20.201282 26760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:20.202445 26760 hybrid_clock.cc:648] HybridClock initialized: now 1785990920202420 us; error 33 us; skew 500 ppm
I20260806 04:35:20.204210 26760 webserver.cc:460] Webserver started at http://127.12.15.194:39131/ using document root <none> and password file <none>
I20260806 04:35:20.204797 26760 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:20.204857 26760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:20.205060 26760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:20.207114 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "45321edfeea94448b4030ad382c3d396"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "6e0d413075851617816a735daec50d32"
server_key_iv: "2eda37890b4d7d34685bc07178c16521"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.207653 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "45321edfeea94448b4030ad382c3d396"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "6e0d413075851617816a735daec50d32"
server_key_iv: "2eda37890b4d7d34685bc07178c16521"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.211355 26760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260806 04:35:20.214017 26781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.215247 26760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:35:20.215358 26760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "45321edfeea94448b4030ad382c3d396"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "6e0d413075851617816a735daec50d32"
server_key_iv: "2eda37890b4d7d34685bc07178c16521"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.215461 26760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:20.230629 26760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:20.231333 26760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:20.231508 26760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:20.232118 26760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:20.233140 26760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:20.233188 26760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.233229 26760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:20.233258 26760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.241796 26760 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:38473
I20260806 04:35:20.241899 26961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:38473 every 8 connection(s)
I20260806 04:35:20.242740 26760 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:35:20.251624 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.244510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:57263 (local address 127.12.15.254:36957)
0806 04:35:20.244717 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:20.244722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:20.245833 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 04:35:20.247192 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:20.247194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:20.247243 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:20.247318 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:20.248216 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.248805 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.249961 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.250229 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.250310 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:20.251265 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:20.251268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:20.251270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:20.251281 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:20.251298 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:20.251305 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:20.251359 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:20.251459 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 04:35:20.251504 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 04:35:20.252516 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.245017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36957 (local address 127.12.15.194:57263)
0806 04:35:20.245682 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:20.245735 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:20.247001 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:20.247465 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:20.247471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:20.247552 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:20.248084 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:20.248100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.248918 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.248921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:20.249855 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:20.249865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.250363 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.250366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:20.250504 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:20.251041 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:35:20.251042 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:20.251079 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:35:20.251138 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:20.251364 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:20.251369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:20.251372 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 04:35:20.251456 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:35:20.251664 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":118,"threads_started":1}
I20260806 04:35:20.252664 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 26760
I20260806 04:35:20.252775 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:35:20.253041 12351 external_mini_cluster.cc:1428] Setting key 44276b1a5faf3c3dab40597784ef2718
I20260806 04:35:20.253693 26962 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36957
I20260806 04:35:20.253916 26962 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:20.254480 26962 heartbeater.cc:511] Master 127.12.15.254:36957 requested a full tablet report, sending...
I20260806 04:35:20.254980 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36957
--builtin_ntp_servers=127.12.15.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 04:35:20.255532 26453 ts_manager.cc:194] Registered new tserver with Master: 45321edfeea94448b4030ad382c3d396 (127.12.15.194:38473)
I20260806 04:35:20.256098 26453 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.194:57263
W20260806 04:35:20.368984 26965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:20.369256 26965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:20.369308 26965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:20.373490 26965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:20.373557 26965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:20.373629 26965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:35:20.378445 26965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:20.379555 26965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:20.380992 26965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:20.390478 26975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.390493 26976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.390472 26979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.390663 26965 server_base.cc:1034] running on GCE node
I20260806 04:35:20.391157 26965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:20.391789 26965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:20.392938 26965 hybrid_clock.cc:648] HybridClock initialized: now 1785990920392929 us; error 32 us; skew 500 ppm
I20260806 04:35:20.394742 26965 webserver.cc:460] Webserver started at http://127.12.15.195:39587/ using document root <none> and password file <none>
I20260806 04:35:20.395301 26965 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:20.395367 26965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:20.395584 26965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:20.397651 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "30aeffe0c88642ef9d75e47c173455a4"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "d182e735020efd481c3b1a10928e937b"
server_key_iv: "52f74678f5e3df3477a860dd3ee8b37f"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.398190 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "30aeffe0c88642ef9d75e47c173455a4"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "d182e735020efd481c3b1a10928e937b"
server_key_iv: "52f74678f5e3df3477a860dd3ee8b37f"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.401938 26965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:35:20.404409 26992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.405606 26965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:35:20.405714 26965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "30aeffe0c88642ef9d75e47c173455a4"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "d182e735020efd481c3b1a10928e937b"
server_key_iv: "52f74678f5e3df3477a860dd3ee8b37f"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.405812 26965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:20.426668 26965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:20.427418 26965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:20.427606 26965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:20.428239 26965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:20.429262 26965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:20.429313 26965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.429369 26965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:20.429410 26965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.438050 26965 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:35369
I20260806 04:35:20.438122 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:35369 every 8 connection(s)
I20260806 04:35:20.439052 26965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:35:20.442031 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 26965
I20260806 04:35:20.442139 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:35:20.442481 12351 external_mini_cluster.cc:1428] Setting key fba8cd1f2824d7623611303ab8a4b951
I20260806 04:35:20.448567 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.441052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33797 (local address 127.12.15.254:36957)
0806 04:35:20.441224 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:20.441229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:20.442416 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 04:35:20.443979 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:20.443982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:20.444026 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:20.444077 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:20.445131 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.445678 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.446771 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.447001 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.447090 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:20.448134 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:20.448137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:20.448139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:20.448151 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:20.448171 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:20.448179 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:20.448237 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:20.448376 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 04:35:20.448450 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 04:35:20.449440 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.441463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36957 (local address 127.12.15.195:33797)
0806 04:35:20.442237 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:20.442295 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:20.443796 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:20.444273 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:20.444281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:20.444418 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:20.444997 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:20.445012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.445772 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.445774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:20.446647 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:20.446656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:20.447139 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:20.447142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:20.447286 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:20.447854 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:35:20.447856 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:20.447897 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:35:20.447986 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:20.448243 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:20.448248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:20.448251 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 04:35:20.448387 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 04:35:20.448640 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260806 04:35:20.450505 27175 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36957
I20260806 04:35:20.450783 27175 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:20.451288 27175 heartbeater.cc:511] Master 127.12.15.254:36957 requested a full tablet report, sending...
I20260806 04:35:20.452389 26453 ts_manager.cc:194] Registered new tserver with Master: 30aeffe0c88642ef9d75e47c173455a4 (127.12.15.195:35369)
I20260806 04:35:20.452978 26453 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.195:33797
I20260806 04:35:20.456291 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 04:35:20.459038 26516 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:20.458169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33580 (local address 127.12.15.254:36957)
0806 04:35:20.458294 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:20.458298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:20.458504 (+ 206us) server_negotiation.cc:407] Connection header received
0806 04:35:20.458595 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:20.458597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:20.458637 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:20.458699 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:20.458917 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33580: BlockingRecv error: recv got EOF from 127.0.0.1:33580 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:35:20.459235 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 26550
I20260806 04:35:20.481869 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 26760
I20260806 04:35:20.497982 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 26965
I20260806 04:35:20.508245 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 26401
2026-08-06T04:35:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (925 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T04:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:35:20Z Disabled control of system clock
I20260806 04:35:20.541942 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38177
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:38177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 04:35:20.652833 27191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:20.653082 27191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:20.653118 27191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:20.657050 27191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:35:20.657105 27191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:20.657119 27191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:20.657135 27191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:35:20.657147 27191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:35:20.661892 27191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:38177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:20.662973 27191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:20.664328 27191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:20.673079 27197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.673045 27198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.673079 27200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.673146 27191 server_base.cc:1034] running on GCE node
I20260806 04:35:20.673946 27191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:20.675055 27191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:20.676242 27191 hybrid_clock.cc:648] HybridClock initialized: now 1785990920676221 us; error 33 us; skew 500 ppm
I20260806 04:35:20.678246 27191 webserver.cc:460] Webserver started at http://127.12.15.254:35515/ using document root <none> and password file <none>
I20260806 04:35:20.678824 27191 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:20.678893 27191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:20.679102 27191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:20.681209 27191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "11c60cb8d05f41e09e73af8090752d38"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "b33ec1fbbbe3a86aede7a0194262711c"
server_key_iv: "1f91b0350acb280fb2b1ef1a45d18e64"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.681764 27191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "11c60cb8d05f41e09e73af8090752d38"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "b33ec1fbbbe3a86aede7a0194262711c"
server_key_iv: "1f91b0350acb280fb2b1ef1a45d18e64"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.685428 27191 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:35:20.687739 27206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.689033 27191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:35:20.689152 27191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "11c60cb8d05f41e09e73af8090752d38"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "b33ec1fbbbe3a86aede7a0194262711c"
server_key_iv: "1f91b0350acb280fb2b1ef1a45d18e64"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.689245 27191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:20.693748 27191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:20.694339 27191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:20.694500 27191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:20.700963 27191 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:38177
I20260806 04:35:20.700966 27287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:38177 every 8 connection(s)
I20260806 04:35:20.701910 27191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:35:20.705312 27288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:20.707319 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 27191
I20260806 04:35:20.707468 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:35:20.707724 12351 external_mini_cluster.cc:1428] Setting key 9914ebd191c98240c7cd8a3368485b36
I20260806 04:35:20.712064 27288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38: Bootstrap starting.
I20260806 04:35:20.713901 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.709238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.12.15.254:38177)
0806 04:35:20.709928 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:20.709950 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:20.709994 (+ 44us) server_negotiation.cc:407] Connection header received
0806 04:35:20.710880 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:20.710904 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:20.711365 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:20.711736 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:20.712451 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:20.712456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:20.712471 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:20.712482 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:20.712549 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:20.712570 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:20.712709 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:20.712775 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 04:35:20.713010 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":109,"threads_started":1}
I20260806 04:35:20.714680 27288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:20.715569 27288 log.cc:824] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:20.717536 27288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38: No bootstrap required, opened a new log
I20260806 04:35:20.721202 27288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c60cb8d05f41e09e73af8090752d38" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38177 } }
I20260806 04:35:20.721431 27288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:20.721470 27288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c60cb8d05f41e09e73af8090752d38, State: Initialized, Role: FOLLOWER
I20260806 04:35:20.721853 27288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c60cb8d05f41e09e73af8090752d38" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38177 } }
I20260806 04:35:20.721983 27288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:35:20.722028 27288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:35:20.722098 27288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:20.723101 27288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c60cb8d05f41e09e73af8090752d38" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38177 } }
I20260806 04:35:20.723404 27288 leader_election.cc:302] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11c60cb8d05f41e09e73af8090752d38; no voters:
I20260806 04:35:20.723685 27288 leader_election.cc:288] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:35:20.723820 27297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:35:20.724014 27297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [term 1 LEADER]: Becoming Leader. State: Replica: 11c60cb8d05f41e09e73af8090752d38, State: Running, Role: LEADER
I20260806 04:35:20.724323 27297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c60cb8d05f41e09e73af8090752d38" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38177 } }
I20260806 04:35:20.724892 27288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:35:20.726022 27298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11c60cb8d05f41e09e73af8090752d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c60cb8d05f41e09e73af8090752d38" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38177 } } }
I20260806 04:35:20.726171 27298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:20.726022 27299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11c60cb8d05f41e09e73af8090752d38. Latest consensus state: current_term: 1 leader_uuid: "11c60cb8d05f41e09e73af8090752d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11c60cb8d05f41e09e73af8090752d38" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38177 } } }
I20260806 04:35:20.726266 27299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38 [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:20.726553 27309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:35:20.729715 27309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:35:20.735164 27309 catalog_manager.cc:1347] Generated new cluster ID: 668fe58fa47640719f65e5423f245420
I20260806 04:35:20.735242 27309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:35:20.775476 27309 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:35:20.775854 27309 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:35:20.784698 27309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11c60cb8d05f41e09e73af8090752d38: Generated new TSK 0
I20260806 04:35:20.785586 27309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:35:20.791877 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38177
--builtin_ntp_servers=127.12.15.212:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 04:35:20.908254 27323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:20.908548 27323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:20.908587 27323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:20.912819 27323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:20.912889 27323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:20.912946 27323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:35:20.918349 27323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:20.919443 27323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:20.920866 27323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:20.930722 27331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:20.930742 27332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.930804 27323 server_base.cc:1034] running on GCE node
W20260806 04:35:20.930742 27334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:20.931401 27323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:20.932082 27323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:20.933264 27323 hybrid_clock.cc:648] HybridClock initialized: now 1785990920933187 us; error 86 us; skew 500 ppm
I20260806 04:35:20.935168 27323 webserver.cc:460] Webserver started at http://127.12.15.193:34481/ using document root <none> and password file <none>
I20260806 04:35:20.935758 27323 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:20.935827 27323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:20.936041 27323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:20.938107 27323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "44c7ea11237f44449b8e8ab521398037"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "9f5587cd92c55824cd9efa6dd7436f42"
server_key_iv: "ba78c61c7ec0b9da980fba79b7952396"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.938663 27323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "44c7ea11237f44449b8e8ab521398037"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "9f5587cd92c55824cd9efa6dd7436f42"
server_key_iv: "ba78c61c7ec0b9da980fba79b7952396"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.942272 27323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:35:20.944815 27341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.946113 27323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:35:20.946245 27323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "44c7ea11237f44449b8e8ab521398037"
format_stamp: "Formatted at 2026-08-06 04:35:20 on dist-test-slave-xt4k"
server_key: "9f5587cd92c55824cd9efa6dd7436f42"
server_key_iv: "ba78c61c7ec0b9da980fba79b7952396"
server_key_version: "encryptionkey@0"
I20260806 04:35:20.946363 27323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:20.951097 27323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:20.951661 27323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:20.951794 27323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:20.952270 27323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:20.953154 27323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:20.953200 27323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.953236 27323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:20.953255 27323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:20.961236 27323 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:45781
I20260806 04:35:20.961284 27511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:45781 every 8 connection(s)
I20260806 04:35:20.962082 27323 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:35:20.968122 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.963862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:37635 (local address 127.12.15.254:38177)
0806 04:35:20.963989 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:20.963993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:20.965223 (+ 1230us) server_negotiation.cc:407] Connection header received
0806 04:35:20.966542 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:20.966545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:20.966601 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:20.966684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:20.967695 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:20.967699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:20.967702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:20.967714 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:20.967733 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:20.967741 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:20.967818 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:20.967939 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 04:35:20.967984 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:35:20.968842 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 27323
I20260806 04:35:20.968952 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:35:20.968986 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:20.964243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38177 (local address 127.12.15.193:37635)
0806 04:35:20.965051 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:20.965105 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:20.966337 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:20.966852 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:20.966859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:20.966945 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:20.967448 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:35:20.967449 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:20.967487 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:35:20.967553 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:20.967808 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:20.967817 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:20.967823 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:35:20.967976 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 04:35:20.968230 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":119,"threads_started":1}
I20260806 04:35:20.969282 12351 external_mini_cluster.cc:1428] Setting key b57fade7b8ef720ee7b4d047fd694568
I20260806 04:35:20.970220 27512 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38177
I20260806 04:35:20.970458 27512 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:20.970892 27512 heartbeater.cc:511] Master 127.12.15.254:38177 requested a full tablet report, sending...
I20260806 04:35:20.971217 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38177
--builtin_ntp_servers=127.12.15.212:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 04:35:20.972409 27233 ts_manager.cc:194] Registered new tserver with Master: 44c7ea11237f44449b8e8ab521398037 (127.12.15.193:45781)
W20260806 04:35:21.085259 27516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:21.085584 27516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:21.085636 27516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:21.089735 27516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:21.089795 27516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:21.089852 27516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:35:21.094769 27516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:21.095875 27516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:21.097306 27516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:21.106827 27524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:21.106868 27516 server_base.cc:1034] running on GCE node
W20260806 04:35:21.106803 27529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:21.106822 27525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:21.107458 27516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:21.108064 27516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:21.109229 27516 hybrid_clock.cc:648] HybridClock initialized: now 1785990921109197 us; error 46 us; skew 500 ppm
I20260806 04:35:21.111006 27516 webserver.cc:460] Webserver started at http://127.12.15.194:38821/ using document root <none> and password file <none>
I20260806 04:35:21.111572 27516 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:21.111635 27516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:21.111826 27516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:21.113922 27516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "4c8702b02c3f4f559896f628d70244b5"
format_stamp: "Formatted at 2026-08-06 04:35:21 on dist-test-slave-xt4k"
server_key: "c4a914eadc954dd7ddc50ae68799668a"
server_key_iv: "7c3d2c7626adb7d447b180cfa46dd9b2"
server_key_version: "encryptionkey@0"
I20260806 04:35:21.114449 27516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "4c8702b02c3f4f559896f628d70244b5"
format_stamp: "Formatted at 2026-08-06 04:35:21 on dist-test-slave-xt4k"
server_key: "c4a914eadc954dd7ddc50ae68799668a"
server_key_iv: "7c3d2c7626adb7d447b180cfa46dd9b2"
server_key_version: "encryptionkey@0"
I20260806 04:35:21.118103 27516 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 04:35:21.120656 27539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:21.121843 27516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:35:21.121942 27516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "4c8702b02c3f4f559896f628d70244b5"
format_stamp: "Formatted at 2026-08-06 04:35:21 on dist-test-slave-xt4k"
server_key: "c4a914eadc954dd7ddc50ae68799668a"
server_key_iv: "7c3d2c7626adb7d447b180cfa46dd9b2"
server_key_version: "encryptionkey@0"
I20260806 04:35:21.122061 27516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:21.127068 27516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:21.127698 27516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:21.127837 27516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:21.128327 27516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:21.129297 27516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:21.129344 27516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:21.129381 27516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:21.129401 27516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:21.137734 27516 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:36907
I20260806 04:35:21.137763 27713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:36907 every 8 connection(s)
I20260806 04:35:21.138607 27516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:35:21.144413 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:21.140372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:34029 (local address 127.12.15.254:38177)
0806 04:35:21.140508 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:21.140512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:21.141521 (+ 1009us) server_negotiation.cc:407] Connection header received
0806 04:35:21.142834 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:21.142837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:21.142883 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:21.142964 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:21.143958 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:21.143962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:21.143965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:21.143978 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:21.143999 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:21.144013 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:21.144091 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:21.144208 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 04:35:21.144253 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:35:21.145229 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:21.140742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38177 (local address 127.12.15.194:34029)
0806 04:35:21.141382 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:21.141420 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:21.142630 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:21.143130 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:21.143137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:21.143222 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:21.143709 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:35:21.143711 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:21.143749 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:35:21.143816 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:21.144080 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:21.144088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:21.144094 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:35:21.144245 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 04:35:21.144543 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260806 04:35:21.146291 27714 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38177
I20260806 04:35:21.146505 27714 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:21.146890 27714 heartbeater.cc:511] Master 127.12.15.254:38177 requested a full tablet report, sending...
I20260806 04:35:21.147879 27233 ts_manager.cc:194] Registered new tserver with Master: 4c8702b02c3f4f559896f628d70244b5 (127.12.15.194:36907)
I20260806 04:35:21.148825 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 27516
I20260806 04:35:21.148947 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:35:21.149257 12351 external_mini_cluster.cc:1428] Setting key ee833ec0f6bf67fdf7ef20ccadb34ca0
I20260806 04:35:21.151041 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38177
--builtin_ntp_servers=127.12.15.212:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 04:35:21.265658 27719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:21.265929 27719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:21.265967 27719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:21.270071 27719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:21.270124 27719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:21.270180 27719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:35:21.274973 27719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:21.276074 27719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:21.277547 27719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:21.286926 27728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:21.287024 27732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:21.287086 27729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:21.287402 27719 server_base.cc:1034] running on GCE node
I20260806 04:35:21.287825 27719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:21.288511 27719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:21.289685 27719 hybrid_clock.cc:648] HybridClock initialized: now 1785990921289665 us; error 31 us; skew 500 ppm
I20260806 04:35:21.291555 27719 webserver.cc:460] Webserver started at http://127.12.15.195:38243/ using document root <none> and password file <none>
I20260806 04:35:21.292148 27719 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:21.292219 27719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:21.292467 27719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:21.294629 27719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "6d4a39f9c063470c8eb6a97ae65a5e87"
format_stamp: "Formatted at 2026-08-06 04:35:21 on dist-test-slave-xt4k"
server_key: "4cbc0756a5c1de5df2fc018b33a1349d"
server_key_iv: "612b9c9f5a7fd96efce395d796f9532b"
server_key_version: "encryptionkey@0"
I20260806 04:35:21.295177 27719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "6d4a39f9c063470c8eb6a97ae65a5e87"
format_stamp: "Formatted at 2026-08-06 04:35:21 on dist-test-slave-xt4k"
server_key: "4cbc0756a5c1de5df2fc018b33a1349d"
server_key_iv: "612b9c9f5a7fd96efce395d796f9532b"
server_key_version: "encryptionkey@0"
I20260806 04:35:21.298929 27719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 04:35:21.301506 27740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:21.302734 27719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:35:21.302855 27719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "6d4a39f9c063470c8eb6a97ae65a5e87"
format_stamp: "Formatted at 2026-08-06 04:35:21 on dist-test-slave-xt4k"
server_key: "4cbc0756a5c1de5df2fc018b33a1349d"
server_key_iv: "612b9c9f5a7fd96efce395d796f9532b"
server_key_version: "encryptionkey@0"
I20260806 04:35:21.302953 27719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:21.307464 27719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:21.308048 27719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:21.308202 27719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:21.308768 27719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:21.309660 27719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:21.309707 27719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:21.309757 27719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:21.309783 27719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:21.318871 27719 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:34197
I20260806 04:35:21.318904 27924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:34197 every 8 connection(s)
I20260806 04:35:21.319788 27719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:35:21.325872 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:21.321662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:45589 (local address 127.12.15.254:38177)
0806 04:35:21.321794 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:21.321798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:21.322917 (+ 1119us) server_negotiation.cc:407] Connection header received
0806 04:35:21.324223 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:21.324226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:21.324269 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:21.324388 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:21.325464 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:21.325467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:21.325469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:35:21.325481 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:21.325500 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:35:21.325507 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:21.325589 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:21.325701 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 04:35:21.325762 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:35:21.326745 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:21.322076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38177 (local address 127.12.15.195:45589)
0806 04:35:21.322771 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:21.322813 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:21.323982 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:21.324541 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:21.324551 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:21.324642 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:21.325178 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:35:21.325181 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:21.325239 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:35:21.325330 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:21.325573 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:21.325577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:21.325581 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:35:21.325717 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 04:35:21.325933 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":115,"threads_started":1}
I20260806 04:35:21.327945 27925 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38177
I20260806 04:35:21.328060 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 27719
I20260806 04:35:21.328166 27925 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:21.328172 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:35:21.328480 12351 external_mini_cluster.cc:1428] Setting key 66962d7c8febf477d8d62ba1198b1eb7
I20260806 04:35:21.328608 27925 heartbeater.cc:511] Master 127.12.15.254:38177 requested a full tablet report, sending...
I20260806 04:35:21.329701 27233 ts_manager.cc:194] Registered new tserver with Master: 6d4a39f9c063470c8eb6a97ae65a5e87 (127.12.15.195:34197)
I20260806 04:35:21.330832 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 04:35:21.333715 27292 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:35:21.332923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.12.15.254:38177)
0806 04:35:21.333052 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:21.333056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:21.333192 (+ 136us) server_negotiation.cc:407] Connection header received
0806 04:35:21.333285 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:21.333287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:21.333316 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:21.333378 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:21.333599 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58520: BlockingRecv error: recv got EOF from 127.0.0.1:58520 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:35:21.333971 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 27323
I20260806 04:35:21.350368 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 27516
I20260806 04:35:21.366797 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 27719
I20260806 04:35:21.393250 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 27191
2026-08-06T04:35:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (884 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:35:21 dist-test-slave-xt4k krb5kdc[27941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:35:21 dist-test-slave-xt4k krb5kdc[27941](info): set up 2 sockets
Aug 06 04:35:21 dist-test-slave-xt4k krb5kdc[27941](info): commencing operation
krb5kdc: starting...
W20260806 04:35:22.534334 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:35:22 dist-test-slave-xt4k krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:35:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:35:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:35:22.658736 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36933
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:22.770251 27964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:22.770545 27964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:22.770603 27964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:22.774721 27964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:35:22.774794 27964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:22.774821 27964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:22.774842 27964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:35:22.774863 27964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:35:22.780372 27964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:22.781486 27964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:22.782552 27964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:22.791208 27978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:22.791240 27972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:22.791378 27964 server_base.cc:1034] running on GCE node
W20260806 04:35:22.791536 27973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:22.792078 27964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:22.793229 27964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:22.794407 27964 hybrid_clock.cc:648] HybridClock initialized: now 1785990922794382 us; error 40 us; skew 500 ppm
Aug 06 04:35:22 dist-test-slave-xt4k krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990922, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:22.797647 27964 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:35:22.798760 27964 webserver.cc:460] Webserver started at http://127.12.15.254:45375/ using document root <none> and password file <none>
I20260806 04:35:22.799469 27964 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:22.799556 27964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:22.799818 27964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:22.802376 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "446b960d3fc74e5c82c79dbedb7a60fc"
format_stamp: "Formatted at 2026-08-06 04:35:22 on dist-test-slave-xt4k"
server_key: "45934e9a2af844dc03ff795558563051"
server_key_iv: "2d298c1f185dba8290bb7568a12ecb7e"
server_key_version: "encryptionkey@0"
I20260806 04:35:22.802953 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "446b960d3fc74e5c82c79dbedb7a60fc"
format_stamp: "Formatted at 2026-08-06 04:35:22 on dist-test-slave-xt4k"
server_key: "45934e9a2af844dc03ff795558563051"
server_key_iv: "2d298c1f185dba8290bb7568a12ecb7e"
server_key_version: "encryptionkey@0"
I20260806 04:35:22.806753 27964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 04:35:22.809158 27987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:22.810314 27964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:35:22.810431 27964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "446b960d3fc74e5c82c79dbedb7a60fc"
format_stamp: "Formatted at 2026-08-06 04:35:22 on dist-test-slave-xt4k"
server_key: "45934e9a2af844dc03ff795558563051"
server_key_iv: "2d298c1f185dba8290bb7568a12ecb7e"
server_key_version: "encryptionkey@0"
I20260806 04:35:22.810524 27964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:22.822531 27964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:22.825240 27964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:22.825409 27964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:22.831954 27964 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36933
I20260806 04:35:22.832230 28063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36933 every 8 connection(s)
I20260806 04:35:22.832906 27964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:35:22.835487 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 27964
I20260806 04:35:22.835620 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:35:22.835932 12351 external_mini_cluster.cc:1428] Setting key 6fb964b000d26ef629d5537f727c1a7b
I20260806 04:35:22.835990 28064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:22.842422 28064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: Bootstrap starting.
Aug 06 04:35:22 dist-test-slave-xt4k krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:35:22.845084 28064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:22.845882 28064 log.cc:824] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:22.847740 28064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: No bootstrap required, opened a new log
I20260806 04:35:22.849447 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:22.837399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.12.15.254:36933)
0806 04:35:22.837826 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:22.837841 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:22.837877 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:35:22.838755 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:22.838773 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:22.839094 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:22.839389 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:22.840149 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:22.841185 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:22.841992 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:22.842470 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:22.842633 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:22.845220 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:22.845226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:22.845240 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:22.845267 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:22.846804 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:22.847207 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:22.847213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:22.847218 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:22.847325 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:22.847586 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:22.847590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:22.847592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:22.848003 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:22.848201 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:22.848487 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 04:35:22.848763 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":118,"threads_started":1}
I20260806 04:35:22.851524 28064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36933 } }
I20260806 04:35:22.851804 28064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:22.851866 28064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446b960d3fc74e5c82c79dbedb7a60fc, State: Initialized, Role: FOLLOWER
I20260806 04:35:22.852281 28064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36933 } }
I20260806 04:35:22.852451 28064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:35:22.852516 28064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:35:22.852622 28064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:22.853447 28064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36933 } }
I20260806 04:35:22.853761 28064 leader_election.cc:302] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 446b960d3fc74e5c82c79dbedb7a60fc; no voters:
I20260806 04:35:22.854036 28064 leader_election.cc:288] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:35:22.854141 28070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:35:22.854350 28070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [term 1 LEADER]: Becoming Leader. State: Replica: 446b960d3fc74e5c82c79dbedb7a60fc, State: Running, Role: LEADER
I20260806 04:35:22.854686 28070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36933 } }
I20260806 04:35:22.855146 28064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:35:22.856185 28074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36933 } } }
I20260806 04:35:22.856362 28074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [sys.catalog]: This master's current role is: LEADER
I20260806 04:35:22.856840 28075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 446b960d3fc74e5c82c79dbedb7a60fc. Latest consensus state: current_term: 1 leader_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "446b960d3fc74e5c82c79dbedb7a60fc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36933 } } }
I20260806 04:35:22.856936 28075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc [sys.catalog]: This master's current role is: LEADER
W20260806 04:35:22.859710 28094 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:35:22.859778 28094 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:35:22.859854 28085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:35:22.860699 28085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:35:22.865855 28085 catalog_manager.cc:1347] Generated new cluster ID: eabe542d42c84666b71306e68f8182c7
I20260806 04:35:22.865918 28085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:35:22.880155 28085 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:35:22.881502 28085 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:35:22.892580 28085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: Generated new TSK 0
I20260806 04:35:22.893468 28085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:35:22.923322 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36933
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:23.037505 28106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:23.037781 28106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:23.037827 28106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:23.041890 28106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:23.041960 28106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:23.042021 28106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:35:23.047398 28106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:23.048553 28106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:23.049582 28106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:23.059227 28116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:23.059237 28117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:23.059221 28120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:23.059500 28106 server_base.cc:1034] running on GCE node
I20260806 04:35:23.059993 28106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:23.060678 28106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:23.061861 28106 hybrid_clock.cc:648] HybridClock initialized: now 1785990923061837 us; error 30 us; skew 500 ppm
Aug 06 04:35:23 dist-test-slave-xt4k krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990923, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:23.064881 28106 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:35:23.065824 28106 webserver.cc:460] Webserver started at http://127.12.15.193:46269/ using document root <none> and password file <none>
I20260806 04:35:23.066381 28106 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:23.066449 28106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:23.066668 28106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:23.068753 28106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "9df49b12ba3841e987ae7511f67728bc"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "1b9b07b1018b30663233572d130954ad"
server_key_iv: "e8cd03e0c179927d140dec2f98c06a63"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.069293 28106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "9df49b12ba3841e987ae7511f67728bc"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "1b9b07b1018b30663233572d130954ad"
server_key_iv: "e8cd03e0c179927d140dec2f98c06a63"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.073031 28106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:35:23.075542 28135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.076982 28106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:35:23.077106 28106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "9df49b12ba3841e987ae7511f67728bc"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "1b9b07b1018b30663233572d130954ad"
server_key_iv: "e8cd03e0c179927d140dec2f98c06a63"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.077224 28106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:23.098865 28106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:23.102042 28106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:23.102202 28106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:23.102783 28106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:23.103595 28106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:23.103642 28106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.103682 28106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:23.103703 28106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.112522 28106 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:39521
I20260806 04:35:23.112560 28298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:39521 every 8 connection(s)
I20260806 04:35:23.113461 28106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:35:23.120199 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 28106
I20260806 04:35:23.120378 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:35:23.120728 12351 external_mini_cluster.cc:1428] Setting key 31b12d9b2ba11a4c18197d0739237e87
Aug 06 04:35:23 dist-test-slave-xt4k krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990923, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:35:23.128672 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.115397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:52381 (local address 127.12.15.254:36933)
0806 04:35:23.115547 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.115553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.116464 (+ 911us) server_negotiation.cc:407] Connection header received
0806 04:35:23.117886 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.117889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.117944 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.118028 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:23.119312 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.119900 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.121518 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.121774 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.121887 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.124451 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:23.124454 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:23.124457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:23.124483 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:23.126254 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.126893 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.126896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.126897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.126983 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.127393 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.127396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.127397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.127611 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:23.127687 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.128414 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.128534 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:35:23.129881 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.115810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36933 (local address 127.12.15.193:52381)
0806 04:35:23.116283 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.116324 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.117676 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.118242 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.118249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.118624 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:23.119141 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.119156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.120063 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.120067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.121350 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.121363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.121986 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.121992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.122143 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.122754 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:23.122771 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:23.124241 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:23.126393 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:23.126400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:23.126419 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:23.126776 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:23.127080 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:23.127084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:23.127086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:23.127292 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:23.127711 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:23.127719 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:23.128083 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.128312 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.128746 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":113,"threads_started":1}
I20260806 04:35:23.131305 28302 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36933
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:35:23.131552 28302 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:23.132066 28302 heartbeater.cc:511] Master 127.12.15.254:36933 requested a full tablet report, sending...
I20260806 04:35:23.133673 28010 ts_manager.cc:194] Registered new tserver with Master: 9df49b12ba3841e987ae7511f67728bc (127.12.15.193:39521)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:35:23.135114 28010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:52381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:35:23.152173 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36933
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:23.264616 28312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:23.264868 28312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:23.264925 28312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:23.268849 28312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:23.268906 28312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:23.268985 28312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:35:23.274122 28312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:23.275126 28312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:23.276120 28312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:23.285703 28327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:23.285750 28324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:23.285714 28325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:23.285813 28312 server_base.cc:1034] running on GCE node
I20260806 04:35:23.286355 28312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:23.286978 28312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:23.288141 28312 hybrid_clock.cc:648] HybridClock initialized: now 1785990923288126 us; error 26 us; skew 500 ppm
Aug 06 04:35:23 dist-test-slave-xt4k krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990923, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:23.295439 28312 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:35:23.296558 28312 webserver.cc:460] Webserver started at http://127.12.15.194:44763/ using document root <none> and password file <none>
I20260806 04:35:23.297138 28312 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:23.297210 28312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:23.297415 28312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:23.299527 28312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "05b6c99342904e3b87a26c4fcd24346b"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "b36b728c7e35437cfd1cb0851c315963"
server_key_iv: "cd797f15ef3ee84c874f78a519d926fd"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.300072 28312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "05b6c99342904e3b87a26c4fcd24346b"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "b36b728c7e35437cfd1cb0851c315963"
server_key_iv: "cd797f15ef3ee84c874f78a519d926fd"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.303855 28312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 04:35:23.306450 28336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.307803 28312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:35:23.307932 28312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "05b6c99342904e3b87a26c4fcd24346b"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "b36b728c7e35437cfd1cb0851c315963"
server_key_iv: "cd797f15ef3ee84c874f78a519d926fd"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.308037 28312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:23.323621 28312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:23.326769 28312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:23.326953 28312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:23.327608 28312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:23.328624 28312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:23.328688 28312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.328747 28312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:23.328773 28312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.337507 28312 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:44859
I20260806 04:35:23.337531 28511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:44859 every 8 connection(s)
I20260806 04:35:23.338369 28312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:35:23.339385 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 28312
I20260806 04:35:23.339507 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:35:23.339833 12351 external_mini_cluster.cc:1428] Setting key 994158a6541f6956d7369aaf361b7349
Aug 06 04:35:23 dist-test-slave-xt4k krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990923, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:35:23.353499 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.340317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:54651 (local address 127.12.15.254:36933)
0806 04:35:23.340462 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.340467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.341424 (+ 957us) server_negotiation.cc:407] Connection header received
0806 04:35:23.342824 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.342827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.342883 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.342957 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:23.344473 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.345013 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.346617 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.346921 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.347010 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.350108 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:23.350112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:23.350114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:23.350140 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:23.351447 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.351970 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.351973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.351974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.352053 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.352416 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.352419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.352420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.352621 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:23.352706 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.353238 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.353354 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:35:23.354415 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.340806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36933 (local address 127.12.15.194:54651)
0806 04:35:23.341269 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.341326 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.342621 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.343132 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.343138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.343639 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:23.344290 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.344309 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.345159 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.345162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.346466 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.346478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.347084 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.347089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.347289 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.347928 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:23.347951 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:23.349900 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:23.351596 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:23.351602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:23.351618 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:23.351866 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:23.352129 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:23.352132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:23.352133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:23.352274 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:23.352729 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:23.352734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:23.353019 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.353172 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.353492 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":113,"threads_started":1}
I20260806 04:35:23.355772 28513 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36933
I20260806 04:35:23.356053 28513 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:23.356729 28513 heartbeater.cc:511] Master 127.12.15.254:36933 requested a full tablet report, sending...
I20260806 04:35:23.357944 28010 ts_manager.cc:194] Registered new tserver with Master: 05b6c99342904e3b87a26c4fcd24346b (127.12.15.194:44859)
I20260806 04:35:23.358532 28010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:54651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:35:23.371546 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36933
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:35:23.489270 28521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:35:23.489550 28521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:35:23.489612 28521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:35:23.493976 28521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:35:23.494058 28521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:35:23.494144 28521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:35:23.499634 28521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:35:23.500797 28521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:35:23.501816 28521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:35:23.511386 28528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:35:23.511380 28527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:23.511492 28521 server_base.cc:1034] running on GCE node
W20260806 04:35:23.511379 28530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:35:23.512084 28521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:35:23.512771 28521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:35:23.513954 28521 hybrid_clock.cc:648] HybridClock initialized: now 1785990923513930 us; error 38 us; skew 500 ppm
Aug 06 04:35:23 dist-test-slave-xt4k krb5kdc[27941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990923, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:35:23.517023 28521 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:35:23.517923 28521 webserver.cc:460] Webserver started at http://127.12.15.195:44697/ using document root <none> and password file <none>
I20260806 04:35:23.518510 28521 fs_manager.cc:359] Metadata directory not provided
I20260806 04:35:23.518575 28521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:35:23.518769 28521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:35:23.520952 28521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "f36334578355450795252abbdc5b2fda"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "7d5c37a52743748f0adf5acf5a876482"
server_key_iv: "4cfb33a260aedd7365e38c7203eebd1d"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.521489 28521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "f36334578355450795252abbdc5b2fda"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "7d5c37a52743748f0adf5acf5a876482"
server_key_iv: "4cfb33a260aedd7365e38c7203eebd1d"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.525578 28521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:35:23.528242 28545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.529570 28521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:35:23.529692 28521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "f36334578355450795252abbdc5b2fda"
format_stamp: "Formatted at 2026-08-06 04:35:23 on dist-test-slave-xt4k"
server_key: "7d5c37a52743748f0adf5acf5a876482"
server_key_iv: "4cfb33a260aedd7365e38c7203eebd1d"
server_key_version: "encryptionkey@0"
I20260806 04:35:23.529798 28521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:35:23.543556 28521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:35:23.546497 28521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:35:23.546665 28521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:35:23.547227 28521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:35:23.548050 28521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:35:23.548097 28521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.548151 28521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:35:23.548177 28521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:35:23.556823 28521 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39503
I20260806 04:35:23.556895 28718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39503 every 8 connection(s)
I20260806 04:35:23.557687 28521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:35:23.559024 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 28521
I20260806 04:35:23.559175 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:35:23.559549 12351 external_mini_cluster.cc:1428] Setting key 57761d8f0d695ea520f570e570ad4ea8
Aug 06 04:35:23 dist-test-slave-xt4k krb5kdc[27941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990923, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:35:23.570766 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.559553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:34699 (local address 127.12.15.254:36933)
0806 04:35:23.559698 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.559702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.560627 (+ 925us) server_negotiation.cc:407] Connection header received
0806 04:35:23.561974 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.561977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.562028 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.562095 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:23.563205 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.563737 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.564922 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.565159 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.565250 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.567637 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:23.567640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:23.567643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:23.567667 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:23.568999 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.569476 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.569479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.569480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.569548 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.569834 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.569836 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.569838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.570028 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:23.570107 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.570536 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.570640 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:35:23.571561 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.559971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36933 (local address 127.12.15.195:34699)
0806 04:35:23.560468 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.560520 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.561794 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.562240 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.562246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.562614 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:35:23.563076 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.563089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.563854 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.563857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.564791 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.564801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.565277 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.565279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.565408 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.565954 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:35:23.565970 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:35:23.567452 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:35:23.569120 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:23.569125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:23.569138 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:23.569391 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:23.569616 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:35:23.569618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:35:23.569619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:35:23.569747 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:35:23.570116 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:35:23.570120 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:35:23.570348 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.570503 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.570760 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260806 04:35:23.572705 28720 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36933
I20260806 04:35:23.572963 28720 heartbeater.cc:464] Registering TS with master...
I20260806 04:35:23.573477 28720 heartbeater.cc:511] Master 127.12.15.254:36933 requested a full tablet report, sending...
I20260806 04:35:23.574481 28009 ts_manager.cc:194] Registered new tserver with Master: f36334578355450795252abbdc5b2fda (127.12.15.195:39503)
I20260806 04:35:23.575229 28009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:34699
I20260806 04:35:23.585175 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:35:23.594305 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.587120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.12.15.254:36933)
0806 04:35:23.587282 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.587287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.587401 (+ 114us) server_negotiation.cc:407] Connection header received
0806 04:35:23.587501 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.587504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.587547 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.587611 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:35:23.588287 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.588882 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.589710 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.589929 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.590005 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.591264 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:35:23.591267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:35:23.591269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:35:23.591292 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:35:23.592662 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.593088 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.593091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.593092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.593156 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:35:23.593462 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:35:23.593465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:35:23.593467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:35:23.593699 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:35:23.593815 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.594041 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.594142 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:35:23.597296 28009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:35:23.600193 28009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:35:23.614595 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.608960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39503 (local address 127.0.0.1:49152)
0806 04:35:23.609986 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.609998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.610089 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.610616 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.610618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.610635 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:35:23.610746 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.610751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.611842 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.611844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.613084 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.613095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.614003 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.614007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.614122 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.614161 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.614270 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.614457 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"mutex_wait_us":28,"thread_start_us":92,"threads_started":1}
I20260806 04:35:23.614581 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.609394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.12.15.195:39503)
0806 04:35:23.609862 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.609873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.610086 (+ 213us) server_negotiation.cc:407] Connection header received
0806 04:35:23.610186 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.610190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.610331 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.610451 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:35:23.610860 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.611701 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.613207 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.613892 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.613990 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.614159 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.614309 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.614412 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":106,"threads_started":1}
I20260806 04:35:23.615023 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.608611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44859 (local address 127.0.0.1:52376)
0806 04:35:23.609407 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.609459 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.609595 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.610504 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.610507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.610523 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:35:23.610729 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.610737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.612187 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.612190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.613447 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.613459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.614651 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.614655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.614764 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.614788 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.614838 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.614898 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":64,"thread_start_us":81,"threads_started":1}
I20260806 04:35:23.615445 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.608697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:39521 (local address 127.0.0.1:50280)
0806 04:35:23.609836 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.609849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.609923 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.610451 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.610457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.610480 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:35:23.610716 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.610725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.612187 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.612190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.613535 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.613552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.615073 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.615076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.615172 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.615196 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.615254 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.615320 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"mutex_wait_us":143,"spinlock_wait_cycles":10496,"thread_start_us":106,"threads_started":1}
I20260806 04:35:23.615729 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.609295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.12.15.194:44859)
0806 04:35:23.610125 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.610137 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.610158 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:35:23.610225 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.610229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.610368 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.610722 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:35:23.610876 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.612051 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.613595 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.614410 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.615298 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.615435 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.615485 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.615579 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":113,"threads_started":1}
I20260806 04:35:23.615981 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.608903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.12.15.193:39521)
0806 04:35:23.609859 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.609874 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.609915 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:35:23.610069 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.610076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.610284 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.610448 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:35:23.610853 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.612037 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.613662 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.614410 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.614510 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.615698 (+ 1188us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.615751 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.615850 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":102,"threads_started":1}
I20260806 04:35:23.617728 28625 tablet_service.cc:1467] Processing CreateTablet for tablet 7587ca692769449cb3db61e676ddcd8b (DEFAULT_TABLE table=test-table [id=4259d1734e5f4e3eb70b663c27790bf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:35:23.618749 28625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7587ca692769449cb3db61e676ddcd8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:23.618466 28411 tablet_service.cc:1467] Processing CreateTablet for tablet 7587ca692769449cb3db61e676ddcd8b (DEFAULT_TABLE table=test-table [id=4259d1734e5f4e3eb70b663c27790bf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:35:23.618467 28202 tablet_service.cc:1467] Processing CreateTablet for tablet 7587ca692769449cb3db61e676ddcd8b (DEFAULT_TABLE table=test-table [id=4259d1734e5f4e3eb70b663c27790bf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:35:23.619488 28411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7587ca692769449cb3db61e676ddcd8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:23.619490 28202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7587ca692769449cb3db61e676ddcd8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:35:23.625456 28745 tablet_bootstrap.cc:492] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Bootstrap starting.
I20260806 04:35:23.626605 28746 tablet_bootstrap.cc:492] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: Bootstrap starting.
I20260806 04:35:23.627604 28747 tablet_bootstrap.cc:492] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Bootstrap starting.
I20260806 04:35:23.627900 28745 tablet_bootstrap.cc:654] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:23.628819 28746 tablet_bootstrap.cc:654] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:23.629348 28745 log.cc:824] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:23.630053 28746 log.cc:824] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:23.630148 28747 tablet_bootstrap.cc:654] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Neither blocks nor log segments found. Creating new log.
I20260806 04:35:23.631021 28747 log.cc:824] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:35:23.631762 28745 tablet_bootstrap.cc:492] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: No bootstrap required, opened a new log
I20260806 04:35:23.631812 28746 tablet_bootstrap.cc:492] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: No bootstrap required, opened a new log
I20260806 04:35:23.631986 28746 ts_tablet_manager.cc:1397] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:35:23.631979 28745 ts_tablet_manager.cc:1397] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 04:35:23.633015 28747 tablet_bootstrap.cc:492] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: No bootstrap required, opened a new log
I20260806 04:35:23.633198 28747 ts_tablet_manager.cc:1397] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 04:35:23.635295 28746 raft_consensus.cc:348] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.635540 28746 raft_consensus.cc:374] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:23.635593 28746 raft_consensus.cc:729] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df49b12ba3841e987ae7511f67728bc, State: Initialized, Role: FOLLOWER
I20260806 04:35:23.636026 28746 consensus_queue.cc:260] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.636127 28745 raft_consensus.cc:348] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.636423 28745 raft_consensus.cc:374] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:23.636485 28745 raft_consensus.cc:729] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36334578355450795252abbdc5b2fda, State: Initialized, Role: FOLLOWER
I20260806 04:35:23.636791 28302 heartbeater.cc:503] Master 127.12.15.254:36933 was elected leader, sending a full tablet report...
I20260806 04:35:23.636904 28745 consensus_queue.cc:260] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.636996 28746 ts_tablet_manager.cc:1428] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:35:23.637506 28747 raft_consensus.cc:348] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.637713 28747 raft_consensus.cc:374] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:35:23.637764 28747 raft_consensus.cc:729] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b6c99342904e3b87a26c4fcd24346b, State: Initialized, Role: FOLLOWER
I20260806 04:35:23.637867 28745 ts_tablet_manager.cc:1428] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260806 04:35:23.637883 28720 heartbeater.cc:503] Master 127.12.15.254:36933 was elected leader, sending a full tablet report...
I20260806 04:35:23.638203 28747 consensus_queue.cc:260] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.638849 28513 heartbeater.cc:503] Master 127.12.15.254:36933 was elected leader, sending a full tablet report...
I20260806 04:35:23.639123 28747 ts_tablet_manager.cc:1428] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 04:35:23.808584 28721 tablet.cc:2369] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:35:23.839495 28514 tablet.cc:2369] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:35:23.851683 28754 raft_consensus.cc:482] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:35:23.851895 28754 raft_consensus.cc:504] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.853055 28754 leader_election.cc:288] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b6c99342904e3b87a26c4fcd24346b (127.12.15.194:44859), 9df49b12ba3841e987ae7511f67728bc (127.12.15.193:39521)
I20260806 04:35:23.857319 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.853306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44859 (local address 127.12.15.195:54735)
0806 04:35:23.853414 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.853429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.853523 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.853800 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.853803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.853824 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:35:23.853945 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.853952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.854846 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.854850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.856117 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.856128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.856903 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.856906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.857015 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.857038 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.857110 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.857182 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":52}
I20260806 04:35:23.857340 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.853418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:54735 (local address 127.12.15.194:44859)
0806 04:35:23.853534 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.853537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.853551 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:35:23.853599 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.853601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.853654 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.853746 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:35:23.854067 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.854735 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.856241 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.856789 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.856888 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.856964 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.857140 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.857215 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 04:35:23.857450 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.853577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:52575 (local address 127.12.15.193:39521)
0806 04:35:23.853728 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.853732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.854050 (+ 318us) server_negotiation.cc:407] Connection header received
0806 04:35:23.854150 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.854152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.854193 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.854248 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:35:23.854500 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.855077 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.856458 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.856863 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.856980 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.857039 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.857264 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.857342 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:35:23.857496 28756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.853306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:39521 (local address 127.12.15.195:52575)
0806 04:35:23.853956 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.853967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:35:23.854064 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:35:23.854276 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:35:23.854279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:35:23.854296 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:35:23.854410 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.854416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.855183 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.855185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.856330 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:35:23.856365 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.857030 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.857034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:35:23.857131 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.857156 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 04:35:23.857230 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 04:35:23.857298 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":336,"thread_start_us":151,"threads_started":1}
I20260806 04:35:23.858108 28231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7587ca692769449cb3db61e676ddcd8b" candidate_uuid: "f36334578355450795252abbdc5b2fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9df49b12ba3841e987ae7511f67728bc" is_pre_election: true
I20260806 04:35:23.858335 28441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7587ca692769449cb3db61e676ddcd8b" candidate_uuid: "f36334578355450795252abbdc5b2fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b6c99342904e3b87a26c4fcd24346b" is_pre_election: true
I20260806 04:35:23.858443 28231 raft_consensus.cc:2454] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36334578355450795252abbdc5b2fda in term 0.
I20260806 04:35:23.858611 28441 raft_consensus.cc:2454] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f36334578355450795252abbdc5b2fda in term 0.
I20260806 04:35:23.858924 28550 leader_election.cc:302] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9df49b12ba3841e987ae7511f67728bc, f36334578355450795252abbdc5b2fda; no voters:
I20260806 04:35:23.859222 28754 raft_consensus.cc:2781] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:35:23.859306 28754 raft_consensus.cc:482] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:35:23.859336 28754 raft_consensus.cc:3037] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:23.860383 28754 raft_consensus.cc:504] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
I20260806 04:35:23.860733 28754 leader_election.cc:288] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [CANDIDATE]: Term 1 election: Requested vote from peers 05b6c99342904e3b87a26c4fcd24346b (127.12.15.194:44859), 9df49b12ba3841e987ae7511f67728bc (127.12.15.193:39521)
I20260806 04:35:23.861079 28441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7587ca692769449cb3db61e676ddcd8b" candidate_uuid: "f36334578355450795252abbdc5b2fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b6c99342904e3b87a26c4fcd24346b"
I20260806 04:35:23.861110 28231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7587ca692769449cb3db61e676ddcd8b" candidate_uuid: "f36334578355450795252abbdc5b2fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9df49b12ba3841e987ae7511f67728bc"
I20260806 04:35:23.861212 28441 raft_consensus.cc:3037] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:23.861212 28231 raft_consensus.cc:3037] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:35:23.862464 28231 raft_consensus.cc:2454] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f36334578355450795252abbdc5b2fda in term 1.
I20260806 04:35:23.862463 28441 raft_consensus.cc:2454] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f36334578355450795252abbdc5b2fda in term 1.
I20260806 04:35:23.862752 28548 leader_election.cc:302] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b6c99342904e3b87a26c4fcd24346b, f36334578355450795252abbdc5b2fda; no voters:
I20260806 04:35:23.862974 28754 raft_consensus.cc:2781] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:35:23.863184 28754 raft_consensus.cc:686] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 1 LEADER]: Becoming Leader. State: Replica: f36334578355450795252abbdc5b2fda, State: Running, Role: LEADER
I20260806 04:35:23.863526 28754 consensus_queue.cc:237] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } }
W20260806 04:35:23.864750 28303 tablet.cc:2369] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:35:23.867019 28010 catalog_manager.cc:5495] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda reported cstate change: term changed from 0 to 1, leader changed from <none> to f36334578355450795252abbdc5b2fda (127.12.15.195). New cstate: current_term: 1 leader_uuid: "f36334578355450795252abbdc5b2fda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f36334578355450795252abbdc5b2fda" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39503 } health_report { overall_health: HEALTHY } } }
I20260806 04:35:23.916797 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.913151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.12.15.195:39503)
0806 04:35:23.913290 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:35:23.913295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:35:23.913310 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:35:23.913464 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:35:23.913466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:35:23.913527 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:35:23.913635 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:35:23.913892 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.914455 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.915315 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:35:23.915534 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:35:23.915621 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:35:23.915937 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:35:23.916395 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:35:23.916509 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:35:23.916580 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 04:35:23.916637 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 04:35:23.922577 28441 raft_consensus.cc:1261] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 1 FOLLOWER]: Refusing update from remote peer f36334578355450795252abbdc5b2fda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:35:23.922576 28231 raft_consensus.cc:1261] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc [term 1 FOLLOWER]: Refusing update from remote peer f36334578355450795252abbdc5b2fda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:35:23.923234 28760 consensus_queue.cc:1035] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b6c99342904e3b87a26c4fcd24346b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:35:23.923458 28754 consensus_queue.cc:1035] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9df49b12ba3841e987ae7511f67728bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:35:23.931684 28768 mvcc.cc:205] Tried to move back new op lower bound from 7315418824380018688 to 7315418824147492864. Current Snapshot: MvccSnapshot[applied={T|T < 7315418824380018688}]
I20260806 04:35:23.938941 28010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:35:23.939096 28010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:35:23.941355 28010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 446b960d3fc74e5c82c79dbedb7a60fc: Sending DeleteTablet for 3 replicas of tablet 7587ca692769449cb3db61e676ddcd8b
I20260806 04:35:23.942086 28411 tablet_service.cc:1514] Processing DeleteTablet for tablet 7587ca692769449cb3db61e676ddcd8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:35:23 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:52376
I20260806 04:35:23.942094 28625 tablet_service.cc:1514] Processing DeleteTablet for tablet 7587ca692769449cb3db61e676ddcd8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:35:23 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:49152
I20260806 04:35:23.942612 28783 tablet_replica.cc:330] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: stopping tablet replica
I20260806 04:35:23.942625 28202 tablet_service.cc:1514] Processing DeleteTablet for tablet 7587ca692769449cb3db61e676ddcd8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:35:23 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:50280
I20260806 04:35:23.943017 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 28106
I20260806 04:35:23.943056 28784 tablet_replica.cc:330] T 7587ca692769449cb3db61e676ddcd8b P 9df49b12ba3841e987ae7511f67728bc: stopping tablet replica
I20260806 04:35:23.945570 28782 tablet_replica.cc:330] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: stopping tablet replica
I20260806 04:35:23.945928 28782 raft_consensus.cc:2229] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:35:23.946341 28782 raft_consensus.cc:2258] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:35:23.943040 28783 raft_consensus.cc:2229] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:35:23.949380 28783 raft_consensus.cc:2258] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:35:23.954067 28782 ts_tablet_manager.cc:1905] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:35:23.955257 28783 ts_tablet_manager.cc:1905] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:35:23.959609 28783 ts_tablet_manager.cc:1918] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:35:23.961308 28782 ts_tablet_manager.cc:1918] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:35:23.961499 28783 log.cc:1196] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/7587ca692769449cb3db61e676ddcd8b
I20260806 04:35:23.961644 28782 log.cc:1196] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/7587ca692769449cb3db61e676ddcd8b
I20260806 04:35:23.962083 28783 ts_tablet_manager.cc:1939] T 7587ca692769449cb3db61e676ddcd8b P f36334578355450795252abbdc5b2fda: Deleting consensus metadata
I20260806 04:35:23.962460 28782 ts_tablet_manager.cc:1939] T 7587ca692769449cb3db61e676ddcd8b P 05b6c99342904e3b87a26c4fcd24346b: Deleting consensus metadata
I20260806 04:35:23.964003 27991 catalog_manager.cc:4841] TS 05b6c99342904e3b87a26c4fcd24346b (127.12.15.194:44859): tablet 7587ca692769449cb3db61e676ddcd8b (table test-table [id=4259d1734e5f4e3eb70b663c27790bf2]) successfully deleted
I20260806 04:35:23.965472 27989 catalog_manager.cc:4841] TS f36334578355450795252abbdc5b2fda (127.12.15.195:39503): tablet 7587ca692769449cb3db61e676ddcd8b (table test-table [id=4259d1734e5f4e3eb70b663c27790bf2]) successfully deleted
W20260806 04:35:23.975749 27992 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:39521 (error 108)
I20260806 04:35:23.976112 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 28312
I20260806 04:35:23.982009 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:35:23.981629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:39521 (local address 127.0.0.1:50296)
0806 04:35:23.981845 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 04:35:23.981917 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:39521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260806 04:35:23.982183 27992 catalog_manager.cc:4584] TS 9df49b12ba3841e987ae7511f67728bc (127.12.15.193:39521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7587ca692769449cb3db61e676ddcd8b: Network error: Client connection negotiation failed: client connection to 127.12.15.193:39521: connect: Connection refused (error 111)
I20260806 04:35:23.989702 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 28521
I20260806 04:35:24.003674 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 27964
2026-08-06T04:35:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2609 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:35:24 dist-test-slave-xt4k krb5kdc[28789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:35:24 dist-test-slave-xt4k krb5kdc[28789](info): set up 2 sockets
Aug 06 04:35:24 dist-test-slave-xt4k krb5kdc[28789](info): commencing operation
krb5kdc: starting...
W20260806 04:35:25.150879 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:35:25 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:35:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:35:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/rangeradmin_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/rangeradmin_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/rangerlookup_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/rangerlookup_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
I20260806 04:35:25.323341 12351 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5_1OQt/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/postgres -l logfile start
2026-08-06 04:35:27.827 UTC [28837] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 04:35:27.827 UTC [28837] LOG: listening on IPv4 address "127.12.15.212", port 35039
2026-08-06 04:35:27.831 UTC [28837] LOG: listening on Unix socket "/tmp/.s.PGSQL.35039"
2026-08-06 04:35:27.860 UTC [28840] LOG: database system was shut down at 2026-08-06 04:35:26 UTC
2026-08-06 04:35:27.863 UTC [28837] LOG: database system is ready to accept connections
I20260806 04:35:28.913183 12351 mini_postgres.cc:96] Postgres bound to 35039
2026-08-06 04:35:28.921 UTC [28854] FATAL: database "slave" does not exist
127.12.15.212:35039 - accepting connections
I20260806 04:35:28.922056 12351 mini_ranger.cc:161] Starting Ranger...
I20260806 04:35:28.938099 12351 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 04:35:29.415839 12351 mini_ranger.cc:87] Created ranger Postgres database
I20260806 04:35:29.415922 12351 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-admin
2026-08-06 04:35:29,737 [I] DB FLAVOR :POSTGRES
2026-08-06 04:35:29,738 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:35:29,738 [I] Checking connection..
2026-08-06 04:35:29,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 04:35:30,002 [I] Checking connection passed.
2026-08-06 04:35:30,003 [I] --------- Verifying version history table ---------
2026-08-06 04:35:30,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:35:30,251 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:35:30,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:35:30,513 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:35:30,513 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 04:35:30,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 04:35:30.749 UTC [28954] WARNING: there is no transaction in progress
2026-08-06 04:35:30,757 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 04:35:30,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:35:30,995 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 04:35:30,995 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:35:30,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 04:35:31,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 04:35:31,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xt4k.c.gcp-upstream.internal','N') ;"
2026-08-06 04:35:31,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:35:31,976 [I] Table x_portal_user does not exist in database ranger
2026-08-06 04:35:31,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:35:32,231 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 04:35:32,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:35:32,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:35:32,715 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 04:35:32,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 04:35:33.682 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.700 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.716 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.732 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.746 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.764 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.817 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.824 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.839 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.849 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.858 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.871 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.880 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.887 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.893 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:33.900 UTC [29114] WARNING: there is no transaction in progress
2026-08-06 04:35:34,540 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 04:35:34,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 04:35:34,785 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 04:35:34,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:35:35,029 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 04:35:35,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:35:35,273 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 04:35:35,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:35:35,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:35:35,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xt4k.c.gcp-upstream.internal';"
2026-08-06 04:35:36,028 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 04:35:36,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 04:35:36,261 [I] DB_PATCHES have already been applied
I20260806 04:35:36.267472 12351 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-admin:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5_1OQt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 04:35:36.267539 12351 mini_ranger.cc:193] Using host: 127.12.15.212
I20260806 04:35:36.271854 12351 mini_ranger.cc:227] Ranger admin URL: http://127.12.15.212:51779
Aug 06, 2026 4:35:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:35:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:35:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:35:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:35:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:35:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:35:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 04:35:37.540630 12351 mini_ranger.cc:160] Time spent starting Ranger: real 8.619s user 0.000s sys 0.012s
Aug 06, 2026 4:35:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:35:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 4:35:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:35:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:35:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 04:36:00.126521 12351 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/rangerkms_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/rangerkms_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.15.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 04:36:00 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990960, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 04:36:00.212062 12351 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 04:36:00.222471 12351 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 04:36:01.077338 12351 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 04:36:01.077441 12351 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms
I20260806 04:36:01.077566 12351 mini_ranger_kms.cc:227] Using postgres at 127.12.15.212:35039
2026-08-06 04:36:01,412 [I] DB FLAVOR :POSTGRES
2026-08-06 04:36:01,412 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:36:01,412 [I] Checking connection
2026-08-06 04:36:01,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 04:36:01,667 [I] connection success
2026-08-06 04:36:01,667 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 04:36:01,667 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 04:36:01,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 04:36:01,917 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 04:36:01,917 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:36:01,918 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 04:36:01,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:35039/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 04:36:02,188 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 04:36:02.656319 12351 mini_ranger_kms.cc:310] Created kms service
I20260806 04:36:02.859830 12351 mini_ranger_kms.cc:326] Created kudu user
I20260806 04:36:02.901795 12351 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 04:36:02 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990960, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:03.336642 12351 mini_ranger_kms.cc:384] Added ranger policy
I20260806 04:36:03.336796 12351 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms:/tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 04:36:03.336822 12351 mini_ranger_kms.cc:240] Using host: 127.12.15.212
I20260806 04:36:03.341547 12351 mini_ranger_kms.cc:276] Ranger KMS PID: 29388
I20260806 04:36:03.341698 12351 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.15.212:36105
Aug 06, 2026 4:36:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:36:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:36:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:36:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:36:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:36:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 04:36:05 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785990965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:05.865352 12351 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.653s user 0.013s sys 0.300s
I20260806 04:36:05.865499 12351 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 04:36:05.865610 12351 mini_ranger_kms.cc:401] 127.12.15.212:36105/kms/v1/keys
Aug 06 04:36:06 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.212@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
Aug 06 04:36:06 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.212@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
Aug 06 04:36:08 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:36:08.855428 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39689
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:39689
--ranger_config_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:36105/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:08.978308 29464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:08.978576 29464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:08.978631 29464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:08.983134 29464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:36:08.983217 29464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:08.983245 29464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:08.983268 29464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:36:08.983294 29464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:36:08.989213 29464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.15.212:36105/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:39689
--ranger_config_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:08.990293 29464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:08.991328 29464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:09.001384 29477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:09.001472 29481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:09.001648 29464 server_base.cc:1034] running on GCE node
W20260806 04:36:09.002027 29476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:09.002665 29464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:09.003785 29464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:09.004956 29464 hybrid_clock.cc:648] HybridClock initialized: now 1785990969004936 us; error 37 us; skew 500 ppm
Aug 06 04:36:09 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:09.008056 29464 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:36:09.009052 29464 webserver.cc:460] Webserver started at http://127.12.15.254:32843/ using document root <none> and password file <none>
I20260806 04:36:09.009613 29464 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:09.009675 29464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:09.009893 29464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:09 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:09.072250 29464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "596dca73c1ad4c178aae7ef79436483c"
format_stamp: "Formatted at 2026-08-06 04:36:09 on dist-test-slave-xt4k"
server_key: "bdc52be1dcb5f86a57a8e3122ec7f357"
server_key_iv: "68239d26b1758f1c5bf934142f4a715a"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:09.073194 29464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "596dca73c1ad4c178aae7ef79436483c"
format_stamp: "Formatted at 2026-08-06 04:36:09 on dist-test-slave-xt4k"
server_key: "bdc52be1dcb5f86a57a8e3122ec7f357"
server_key_iv: "68239d26b1758f1c5bf934142f4a715a"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:09.077591 29464 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:36:09.096519 29491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:09.098368 29464 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 04:36:09.098619 29464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "596dca73c1ad4c178aae7ef79436483c"
format_stamp: "Formatted at 2026-08-06 04:36:09 on dist-test-slave-xt4k"
server_key: "bdc52be1dcb5f86a57a8e3122ec7f357"
server_key_iv: "68239d26b1758f1c5bf934142f4a715a"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:09.098785 29464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:09.119289 29464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:09.123100 29464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:09.123335 29464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:09.132918 29464 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:39689
I20260806 04:36:09.132992 29565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:39689 every 8 connection(s)
I20260806 04:36:09.134330 29464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:36:09.138419 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:36:09.139837 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 29464
I20260806 04:36:09.139967 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:36:09.140271 12351 external_mini_cluster.cc:1428] Setting key 97ef01cbf69fd2407d82c93804edd97d
I20260806 04:36:09.154103 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c: Bootstrap starting.
I20260806 04:36:09.159248 29566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c: Neither blocks nor log segments found. Creating new log.
Aug 06 04:36:09 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:09.160550 29566 log.cc:824] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:36:09.163442 29566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c: No bootstrap required, opened a new log
I20260806 04:36:09.167606 29566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596dca73c1ad4c178aae7ef79436483c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39689 } }
I20260806 04:36:09.168182 29566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:36:09.168371 29566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596dca73c1ad4c178aae7ef79436483c, State: Initialized, Role: FOLLOWER
I20260806 04:36:09.168987 29566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596dca73c1ad4c178aae7ef79436483c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39689 } }
I20260806 04:36:09.169255 29566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:36:09.169351 29569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:09.141924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39588 (local address 127.12.15.254:39689)
0806 04:36:09.142624 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:09.142644 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:09.142693 (+ 49us) server_negotiation.cc:407] Connection header received
0806 04:36:09.143601 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:09.143630 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:09.144116 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:09.144599 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:09.152456 (+ 7857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:09.154051 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:09.155312 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:09.156073 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:09.156267 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:09.160870 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:09.160877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:09.160903 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:09.160942 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:09.163504 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:09.163990 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:09.164006 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:09.164012 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:09.164125 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:09.166774 (+ 2649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:09.166779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:09.166781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:09.167386 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:09.167606 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:09.167906 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 04:36:09.168229 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":181,"threads_started":1}
I20260806 04:36:09.169366 29566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:36:09.169903 29566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:36:09.171242 29566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596dca73c1ad4c178aae7ef79436483c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39689 } }
I20260806 04:36:09.171733 29566 leader_election.cc:302] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 596dca73c1ad4c178aae7ef79436483c; no voters:
I20260806 04:36:09.172173 29566 leader_election.cc:288] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:36:09.172273 29573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:36:09.172519 29573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [term 1 LEADER]: Becoming Leader. State: Replica: 596dca73c1ad4c178aae7ef79436483c, State: Running, Role: LEADER
I20260806 04:36:09.172834 29573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596dca73c1ad4c178aae7ef79436483c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39689 } }
I20260806 04:36:09.174017 29566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:36:09.174247 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 596dca73c1ad4c178aae7ef79436483c. Latest consensus state: current_term: 1 leader_uuid: "596dca73c1ad4c178aae7ef79436483c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596dca73c1ad4c178aae7ef79436483c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39689 } } }
I20260806 04:36:09.174448 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [sys.catalog]: This master's current role is: LEADER
I20260806 04:36:09.174739 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "596dca73c1ad4c178aae7ef79436483c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596dca73c1ad4c178aae7ef79436483c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39689 } } }
I20260806 04:36:09.174878 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c [sys.catalog]: This master's current role is: LEADER
I20260806 04:36:09.175343 29578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:36:09.178794 29578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:36:09.179389 29566 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 04:36:09.191974 29578 catalog_manager.cc:1347] Generated new cluster ID: 36694ad872964e40a13917ade2df1dc6
I20260806 04:36:09.192076 29578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:36:09.211963 29578 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:36:09.213459 29578 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:36:09.224504 29578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c: Generated new TSK 0
I20260806 04:36:09.225804 29578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:36:09.967708 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:36105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39689
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:10.111460 29630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:10.111825 29630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:10.111891 29630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:10.117784 29630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:10.117853 29630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:10.117924 29630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:36:10.125567 29630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.15.212:36105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:10.126996 29630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:10.128402 29630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:10.140946 29639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:10.141199 29630 server_base.cc:1034] running on GCE node
W20260806 04:36:10.141321 29643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:10.140918 29638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:10.141894 29630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:10.142555 29630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:10.145848 29630 hybrid_clock.cc:648] HybridClock initialized: now 1785990970145814 us; error 41 us; skew 500 ppm
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:10.149411 29630 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:36:10.150524 29630 webserver.cc:460] Webserver started at http://127.12.15.193:42459/ using document root <none> and password file <none>
I20260806 04:36:10.151376 29630 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:10.151472 29630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:10.151765 29630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:10.169526 29630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "32ada4678d8d4e7fb0a2c1740223fbad"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "3b480c0bbf415707a95c89e4267a8025"
server_key_iv: "514ef5f3b64233f4ef9cb0a7fb02fe45"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.170377 29630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "32ada4678d8d4e7fb0a2c1740223fbad"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "3b480c0bbf415707a95c89e4267a8025"
server_key_iv: "514ef5f3b64233f4ef9cb0a7fb02fe45"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.194278 29630 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.003s sys 0.000s
I20260806 04:36:10.221686 29659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.223518 29630 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260806 04:36:10.223685 29630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "32ada4678d8d4e7fb0a2c1740223fbad"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "3b480c0bbf415707a95c89e4267a8025"
server_key_iv: "514ef5f3b64233f4ef9cb0a7fb02fe45"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.223846 29630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:10.246348 29630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:10.250185 29630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:10.250509 29630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:10.251227 29630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:10.252384 29630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:10.252489 29630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.252615 29630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:10.252692 29630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.264724 29630 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:33239
I20260806 04:36:10.266642 29630 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:36:10.268489 29829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:33239 every 8 connection(s)
I20260806 04:36:10.268965 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 29630
I20260806 04:36:10.269105 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:36:10.269413 12351 external_mini_cluster.cc:1428] Setting key 11622621956b7d2d8376a3ce0c50aa0f
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:36:10.287118 29835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:10.270888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:35809 (local address 127.12.15.254:39689)
0806 04:36:10.273097 (+ 2209us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:10.273102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:10.273116 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:36:10.273159 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:10.273162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:10.273211 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:10.273298 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:10.274896 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.275722 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.277456 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.277739 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.279472 (+ 1733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:10.281089 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:10.281093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:10.281096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:10.281127 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:10.283082 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:10.283796 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:10.283800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:10.283801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:10.283907 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:10.284610 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:10.284613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:10.284616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:10.284873 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:10.284978 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:10.286812 (+ 1834us) server_negotiation.cc:299] Negotiation successful
0806 04:36:10.286951 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2128,"thread_start_us":91,"threads_started":1}
I20260806 04:36:10.287216 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:10.269622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39689 (local address 127.12.15.193:35809)
0806 04:36:10.271465 (+ 1843us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:10.271527 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:10.272667 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:10.273638 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:10.273648 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:10.274020 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:10.274723 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:10.274744 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.275880 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.275884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:10.277332 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:10.277344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.277935 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.277940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:10.278154 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:10.278911 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:10.278934 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:10.280848 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:10.283231 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:10.283238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:10.283259 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:10.283660 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:10.284139 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:10.284142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:10.284145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:10.284390 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:10.285049 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:10.285058 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:10.285455 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 04:36:10.285690 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 04:36:10.286061 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1549,"thread_start_us":233,"threads_started":1}
I20260806 04:36:10.288729 29832 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39689
I20260806 04:36:10.289041 29832 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:36:10.289623 29832 heartbeater.cc:511] Master 127.12.15.254:39689 requested a full tablet report, sending...
I20260806 04:36:10.291275 29514 ts_manager.cc:194] Registered new tserver with Master: 32ada4678d8d4e7fb0a2c1740223fbad (127.12.15.193:33239)
I20260806 04:36:10.292732 29514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:35809
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:36:10.312700 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:36105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39689
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:10.459095 29839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:10.459463 29839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:10.459538 29839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:10.465484 29839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:10.465560 29839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:10.465647 29839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:36:10.473703 29839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.15.212:36105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:10.475219 29839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:10.476673 29839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:10.491184 29856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:10.491550 29852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:10.491741 29850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:10.492211 29839 server_base.cc:1034] running on GCE node
I20260806 04:36:10.492833 29839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:10.493731 29839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:10.495002 29839 hybrid_clock.cc:648] HybridClock initialized: now 1785990970494969 us; error 44 us; skew 500 ppm
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:10.498787 29839 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:36:10.499884 29839 webserver.cc:460] Webserver started at http://127.12.15.194:46295/ using document root <none> and password file <none>
I20260806 04:36:10.500651 29839 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:10.500743 29839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:10.501137 29839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:10.522384 29839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "b87671f56f5b44fbbbdae745e7c52f5f"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "70bec51b3985e0cbca0ef48b11ce985c"
server_key_iv: "c94fcd98fbe17f65306ac9054c961985"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.523171 29839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "b87671f56f5b44fbbbdae745e7c52f5f"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "70bec51b3985e0cbca0ef48b11ce985c"
server_key_iv: "c94fcd98fbe17f65306ac9054c961985"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.528713 29839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 04:36:10.548556 29867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.550361 29839 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 04:36:10.550539 29839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "b87671f56f5b44fbbbdae745e7c52f5f"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "70bec51b3985e0cbca0ef48b11ce985c"
server_key_iv: "c94fcd98fbe17f65306ac9054c961985"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.550694 29839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:10.576920 29839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:10.580895 29839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:10.581173 29839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:10.581959 29839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:10.583148 29839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:10.583257 29839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.583359 29839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:10.583420 29839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.594309 29839 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:46139
I20260806 04:36:10.595721 29839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:36:10.599097 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 29839
I20260806 04:36:10.599233 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:36:10.599550 12351 external_mini_cluster.cc:1428] Setting key 5a94ef3113afcae1e024dea13be4b276
I20260806 04:36:10.606307 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:46139 every 8 connection(s)
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:36:10.621042 29835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:10.607270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:57835 (local address 127.12.15.254:39689)
0806 04:36:10.607406 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:10.607410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:10.608120 (+ 710us) server_negotiation.cc:407] Connection header received
0806 04:36:10.609594 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:10.609598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:10.609663 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:10.609721 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:10.611107 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.611640 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.613471 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.613815 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.613896 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:10.617093 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:10.617097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:10.617100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:10.617131 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:10.618526 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:10.619149 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:10.619153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:10.619155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:10.619253 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:10.619672 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:10.619675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:10.619677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:10.619907 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:10.619990 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:10.620716 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 04:36:10.620857 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:36:10.622948 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:10.607293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39689 (local address 127.12.15.194:57835)
0806 04:36:10.607950 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:10.608007 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:10.609317 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:10.610147 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:10.610154 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:10.610437 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:10.610933 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:10.610946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.611818 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.611823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:10.613323 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:10.613335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.614134 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.614138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:10.614343 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:10.615040 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:10.615065 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:10.616888 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:10.618642 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:10.618649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:10.618666 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:10.619026 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:10.619362 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:10.619365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:10.619367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:10.619560 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:10.620060 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:10.620067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:10.620432 (+ 365us) client_negotiation.cc:769] Sending connection context
0806 04:36:10.621516 (+ 1084us) client_negotiation.cc:240] Negotiation successful
0806 04:36:10.621886 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":189,"threads_started":1}
I20260806 04:36:10.624133 30031 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39689
I20260806 04:36:10.624413 30031 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:10.624934 30031 heartbeater.cc:511] Master 127.12.15.254:39689 requested a full tablet report, sending...
I20260806 04:36:10.625998 29514 ts_manager.cc:194] Registered new tserver with Master: b87671f56f5b44fbbbdae745e7c52f5f (127.12.15.194:46139)
I20260806 04:36:10.626657 29514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:57835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:36:10.643800 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:36105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39689
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:10.796510 30043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:10.796962 30043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:10.797098 30043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:10.803167 30043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:10.803310 30043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:10.803443 30043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:36:10.811298 30043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.15.212:36105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:10.812742 30043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:10.814138 30043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
W20260806 04:36:10.826942 30051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:10.827414 30056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:10.827455 30050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:10.828590 30043 server_base.cc:1034] running on GCE node
I20260806 04:36:10.829087 30043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:10.829883 30043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:10.831096 30043 hybrid_clock.cc:648] HybridClock initialized: now 1785990970831066 us; error 33 us; skew 500 ppm
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:10.834389 30043 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:36:10.835530 30043 webserver.cc:460] Webserver started at http://127.12.15.195:34815/ using document root <none> and password file <none>
I20260806 04:36:10.836290 30043 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:10.836477 30043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:10.836871 30043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:10.852110 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "0a5ee58cd98a4dcc87461422fd606803"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "622b484ac742cc5860543286cc9a4b3e"
server_key_iv: "72528e570390f513515deb1cd7b08fbb"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.853025 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "0a5ee58cd98a4dcc87461422fd606803"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "622b484ac742cc5860543286cc9a4b3e"
server_key_iv: "72528e570390f513515deb1cd7b08fbb"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.874249 30043 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.004s sys 0.000s
I20260806 04:36:10.891894 30067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.893709 30043 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 04:36:10.893851 30043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "0a5ee58cd98a4dcc87461422fd606803"
format_stamp: "Formatted at 2026-08-06 04:36:10 on dist-test-slave-xt4k"
server_key: "622b484ac742cc5860543286cc9a4b3e"
server_key_iv: "72528e570390f513515deb1cd7b08fbb"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:10.893992 30043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:10.924059 30043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:10.927984 30043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:10.928270 30043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:10.929100 30043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:10.930325 30043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:10.930428 30043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.930536 30043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:10.930594 30043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:10.943388 30043 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:38493
I20260806 04:36:10.944772 30043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:36:10.952984 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 30043
I20260806 04:36:10.953135 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:36:10.953457 12351 external_mini_cluster.cc:1428] Setting key 48016260ed68e6724a7e18ace6b06114
I20260806 04:36:10.959065 30232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:38493 every 8 connection(s)
Aug 06 04:36:10 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:10.987373 29835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:10.957198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33649 (local address 127.12.15.254:39689)
0806 04:36:10.957326 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:10.957330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:10.960221 (+ 2891us) server_negotiation.cc:407] Connection header received
0806 04:36:10.961632 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:10.961636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:10.961695 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:10.961751 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:10.963558 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.964431 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.966201 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.966560 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.966640 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:10.970134 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:10.970138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:10.970141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:10.970171 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:10.971835 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:10.985165 (+ 13330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:10.985170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:10.985173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:10.985307 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:10.985875 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:10.985879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:10.985881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:10.986147 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:10.986236 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:10.987049 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 04:36:10.987203 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 04:36:10.989482 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:10.957816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39689 (local address 127.12.15.195:33649)
0806 04:36:10.960055 (+ 2239us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:10.960117 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:10.961362 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:10.962185 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:10.962194 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:10.962615 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:10.963369 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:10.963391 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.964649 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.964652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:10.966061 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:10.966073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:10.966853 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:10.966857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:10.967059 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:10.967856 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:10.967879 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:10.969948 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:10.979027 (+ 9079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:10.979035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:10.979058 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:10.979473 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:10.985434 (+ 5961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:10.985437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:10.985439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:10.985702 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:10.986320 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:10.986328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:10.986728 (+ 400us) client_negotiation.cc:769] Sending connection context
0806 04:36:10.987921 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0806 04:36:10.988368 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1997,"thread_start_us":244,"threads_started":1}
I20260806 04:36:10.991295 30235 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39689
I20260806 04:36:10.991560 30235 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:10.992202 30235 heartbeater.cc:511] Master 127.12.15.254:39689 requested a full tablet report, sending...
I20260806 04:36:10.993427 29514 ts_manager.cc:194] Registered new tserver with Master: 0a5ee58cd98a4dcc87461422fd606803 (127.12.15.195:38493)
I20260806 04:36:10.994197 29514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:33649
I20260806 04:36:10.995077 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:36:11.022338 29835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.004056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.12.15.254:39689)
0806 04:36:11.004226 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.004231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.012913 (+ 8682us) server_negotiation.cc:407] Connection header received
0806 04:36:11.013080 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.013084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.013146 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.013249 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:11.014135 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.014907 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.016102 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.016473 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.016581 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.018476 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:11.018480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:11.018482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:11.018514 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:11.020292 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:11.020865 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:11.020869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:11.020871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:11.020971 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:11.021316 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:11.021320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:11.021322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:11.021565 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:11.021673 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.022027 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.022160 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 04:36:11.026475 29510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:36:11.031287 29510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:36:11.056188 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.043665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:38493 (local address 127.0.0.1:45942)
0806 04:36:11.048685 (+ 5020us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.048695 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:11.048748 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:11.049844 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:11.049850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:11.049872 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:36:11.050019 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.050025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.052163 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.052167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.054291 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.054301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.055642 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.055646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.055745 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.055788 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 04:36:11.055894 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 04:36:11.056025 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3643,"thread_start_us":154,"threads_started":1}
I20260806 04:36:11.057046 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.044721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.12.15.195:38493)
0806 04:36:11.049219 (+ 4498us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.049230 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.049254 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:36:11.049327 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.049332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.049534 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.049676 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:36:11.050147 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.052032 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.054520 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.055326 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.055423 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.055584 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.056786 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.056892 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4388,"thread_start_us":135,"threads_started":1}
I20260806 04:36:11.061091 30121 tablet_service.cc:1467] Processing CreateTablet for tablet bf2c941d613d4a5b9defcdeb82137303 (DEFAULT_TABLE table=test-table [id=11bfb2723348462cb9a5be9bc64d2eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:36:11.062359 30121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2c941d613d4a5b9defcdeb82137303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:36:11.064226 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.046943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:46139 (local address 127.0.0.1:37822)
0806 04:36:11.048901 (+ 1958us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.048911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:11.048974 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:11.059332 (+ 10358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:11.059335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:11.059354 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:36:11.059438 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.059443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.060917 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.060921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.062009 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.062020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.063850 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.063853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.063939 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.063967 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 04:36:11.064027 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 04:36:11.064093 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1903,"thread_start_us":89,"threads_started":1}
I20260806 04:36:11.065124 30268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.047234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.12.15.194:46139)
0806 04:36:11.058864 (+ 11630us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.058879 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.058905 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:36:11.058976 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.058982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.059176 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.059584 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:36:11.059720 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.060787 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.062953 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.063712 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.064675 (+ 963us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.064826 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.064876 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.064963 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11526,"thread_start_us":86,"threads_started":1}
I20260806 04:36:11.068382 29924 tablet_service.cc:1467] Processing CreateTablet for tablet bf2c941d613d4a5b9defcdeb82137303 (DEFAULT_TABLE table=test-table [id=11bfb2723348462cb9a5be9bc64d2eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:36:11.069576 29924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2c941d613d4a5b9defcdeb82137303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:36:11.075183 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.046784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.12.15.193:33239)
0806 04:36:11.054710 (+ 7926us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.054717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.054743 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:36:11.054814 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.054819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.055018 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.055177 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:36:11.055576 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.057168 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.073428 (+ 16260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.074280 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.074370 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.074527 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.074903 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.075019 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7808,"thread_start_us":73,"threads_started":1}
I20260806 04:36:11.075789 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.046601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:33239 (local address 127.0.0.1:47858)
0806 04:36:11.048391 (+ 1790us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.048446 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:11.048569 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:11.055326 (+ 6757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:11.055330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:11.055348 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:36:11.055437 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.055443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.072028 (+ 16585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.072032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.073245 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.073259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.074635 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.074639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.074731 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.074757 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:36:11.075569 (+ 812us) client_negotiation.cc:240] Negotiation successful
0806 04:36:11.075656 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1663,"thread_start_us":91,"threads_started":1}
I20260806 04:36:11.078976 29731 tablet_service.cc:1467] Processing CreateTablet for tablet bf2c941d613d4a5b9defcdeb82137303 (DEFAULT_TABLE table=test-table [id=11bfb2723348462cb9a5be9bc64d2eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:36:11.080111 29731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf2c941d613d4a5b9defcdeb82137303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:36:11.086591 30271 tablet_bootstrap.cc:492] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Bootstrap starting.
I20260806 04:36:11.088380 30270 tablet_bootstrap.cc:492] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Bootstrap starting.
I20260806 04:36:11.090580 30271 tablet_bootstrap.cc:654] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Neither blocks nor log segments found. Creating new log.
I20260806 04:36:11.093420 30271 log.cc:824] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Log is configured to *not* fsync() on all Append() calls
I20260806 04:36:11.093900 30270 tablet_bootstrap.cc:654] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Neither blocks nor log segments found. Creating new log.
I20260806 04:36:11.094807 30270 log.cc:824] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Log is configured to *not* fsync() on all Append() calls
I20260806 04:36:11.095721 30272 tablet_bootstrap.cc:492] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: Bootstrap starting.
I20260806 04:36:11.098255 30272 tablet_bootstrap.cc:654] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: Neither blocks nor log segments found. Creating new log.
I20260806 04:36:11.099277 30272 log.cc:824] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: Log is configured to *not* fsync() on all Append() calls
I20260806 04:36:11.106842 30271 tablet_bootstrap.cc:492] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: No bootstrap required, opened a new log
I20260806 04:36:11.107098 30271 ts_tablet_manager.cc:1397] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.006s
I20260806 04:36:11.111984 30270 tablet_bootstrap.cc:492] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: No bootstrap required, opened a new log
I20260806 04:36:11.112241 30270 ts_tablet_manager.cc:1397] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260806 04:36:11.112970 30271 raft_consensus.cc:348] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.113411 30271 raft_consensus.cc:374] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:36:11.113543 30271 raft_consensus.cc:729] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87671f56f5b44fbbbdae745e7c52f5f, State: Initialized, Role: FOLLOWER
I20260806 04:36:11.114162 30271 consensus_queue.cc:260] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.115454 30271 ts_tablet_manager.cc:1428] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260806 04:36:11.115895 30031 heartbeater.cc:503] Master 127.12.15.254:39689 was elected leader, sending a full tablet report...
I20260806 04:36:11.116523 30270 raft_consensus.cc:348] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.116853 30270 raft_consensus.cc:374] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:36:11.116930 30270 raft_consensus.cc:729] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5ee58cd98a4dcc87461422fd606803, State: Initialized, Role: FOLLOWER
I20260806 04:36:11.117462 30270 consensus_queue.cc:260] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.118551 30270 ts_tablet_manager.cc:1428] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 04:36:11.118930 30235 heartbeater.cc:503] Master 127.12.15.254:39689 was elected leader, sending a full tablet report...
I20260806 04:36:11.129292 30272 tablet_bootstrap.cc:492] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: No bootstrap required, opened a new log
I20260806 04:36:11.129581 30272 ts_tablet_manager.cc:1397] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.000s
I20260806 04:36:11.135289 30272 raft_consensus.cc:348] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.135723 30272 raft_consensus.cc:374] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:36:11.135848 30272 raft_consensus.cc:729] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ada4678d8d4e7fb0a2c1740223fbad, State: Initialized, Role: FOLLOWER
I20260806 04:36:11.136514 30272 consensus_queue.cc:260] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.137781 30272 ts_tablet_manager.cc:1428] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 04:36:11.138298 29832 heartbeater.cc:503] Master 127.12.15.254:39689 was elected leader, sending a full tablet report...
I20260806 04:36:11.158658 30284 raft_consensus.cc:482] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:36:11.158917 30284 raft_consensus.cc:504] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.160363 30284 leader_election.cc:288] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a5ee58cd98a4dcc87461422fd606803 (127.12.15.195:38493), b87671f56f5b44fbbbdae745e7c52f5f (127.12.15.194:46139)
I20260806 04:36:11.170138 30268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.161954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:33597 (local address 127.12.15.194:46139)
0806 04:36:11.162099 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.162103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.165281 (+ 3178us) server_negotiation.cc:407] Connection header received
0806 04:36:11.165514 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.165519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.165574 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.165623 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:36:11.166104 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.166845 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.168808 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.169401 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.169496 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.169553 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.169913 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.170006 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:36:11.170822 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.161254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:46139 (local address 127.12.15.193:33597)
0806 04:36:11.165180 (+ 3926us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.165196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:11.165375 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:11.165775 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:11.165778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:11.165798 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:36:11.165942 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.165961 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.167005 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.167008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.168660 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.168673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.169666 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.169670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.169770 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.169794 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:36:11.170577 (+ 783us) client_negotiation.cc:240] Negotiation successful
0806 04:36:11.170658 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3843,"thread_start_us":98,"threads_started":1}
I20260806 04:36:11.171631 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2c941d613d4a5b9defcdeb82137303" candidate_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" is_pre_election: true
I20260806 04:36:11.172024 29966 raft_consensus.cc:2454] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ada4678d8d4e7fb0a2c1740223fbad in term 0.
I20260806 04:36:11.172838 29663 leader_election.cc:302] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ada4678d8d4e7fb0a2c1740223fbad, b87671f56f5b44fbbbdae745e7c52f5f; no voters:
I20260806 04:36:11.173187 30284 raft_consensus.cc:2781] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:36:11.173264 30284 raft_consensus.cc:482] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:36:11.173303 30284 raft_consensus.cc:3037] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:36:11.174861 30284 raft_consensus.cc:504] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.175262 30284 leader_election.cc:288] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [CANDIDATE]: Term 1 election: Requested vote from peers 0a5ee58cd98a4dcc87461422fd606803 (127.12.15.195:38493), b87671f56f5b44fbbbdae745e7c52f5f (127.12.15.194:46139)
I20260806 04:36:11.175969 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2c941d613d4a5b9defcdeb82137303" candidate_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b87671f56f5b44fbbbdae745e7c52f5f"
I20260806 04:36:11.176095 29966 raft_consensus.cc:3037] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:36:11.177429 29966 raft_consensus.cc:2454] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ada4678d8d4e7fb0a2c1740223fbad in term 1.
I20260806 04:36:11.177853 29663 leader_election.cc:302] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ada4678d8d4e7fb0a2c1740223fbad, b87671f56f5b44fbbbdae745e7c52f5f; no voters:
I20260806 04:36:11.178164 30284 raft_consensus.cc:2781] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:36:11.178375 30284 raft_consensus.cc:686] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 1 LEADER]: Becoming Leader. State: Replica: 32ada4678d8d4e7fb0a2c1740223fbad, State: Running, Role: LEADER
I20260806 04:36:11.178725 30284 consensus_queue.cc:237] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } }
I20260806 04:36:11.181260 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.161812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:50525 (local address 127.12.15.195:38493)
0806 04:36:11.161984 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.161989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.164516 (+ 2527us) server_negotiation.cc:407] Connection header received
0806 04:36:11.177045 (+ 12529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.177049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.177107 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.177163 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:36:11.177583 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.178352 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.179917 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.180555 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.180658 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.180713 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.181051 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.181135 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:36:11.181749 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.160831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:38493 (local address 127.12.15.193:50525)
0806 04:36:11.164412 (+ 3581us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.164429 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:11.176875 (+ 12446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:11.177316 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:11.177319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:11.177338 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:36:11.177451 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.177457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.178527 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.178530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.179772 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:11.179782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.180809 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.180812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:11.180913 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.180936 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:36:11.181513 (+ 577us) client_negotiation.cc:240] Negotiation successful
0806 04:36:11.181593 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3481,"thread_start_us":140,"threads_started":1}
I20260806 04:36:11.182550 30164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2c941d613d4a5b9defcdeb82137303" candidate_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a5ee58cd98a4dcc87461422fd606803" is_pre_election: true
I20260806 04:36:11.182243 29510 catalog_manager.cc:5495] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad reported cstate change: term changed from 0 to 1, leader changed from <none> to 32ada4678d8d4e7fb0a2c1740223fbad (127.12.15.193). New cstate: current_term: 1 leader_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33239 } health_report { overall_health: HEALTHY } } }
I20260806 04:36:11.182874 30164 raft_consensus.cc:2454] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ada4678d8d4e7fb0a2c1740223fbad in term 0.
I20260806 04:36:11.183218 30163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf2c941d613d4a5b9defcdeb82137303" candidate_uuid: "32ada4678d8d4e7fb0a2c1740223fbad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a5ee58cd98a4dcc87461422fd606803"
I20260806 04:36:11.183333 30163 raft_consensus.cc:3037] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:36:11.184837 30163 raft_consensus.cc:2454] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ada4678d8d4e7fb0a2c1740223fbad in term 1.
W20260806 04:36:11.205690 30237 tablet.cc:2369] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:36:11.213004 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.207525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.12.15.193:33239)
0806 04:36:11.207672 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:11.207677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:11.207904 (+ 227us) server_negotiation.cc:407] Connection header received
0806 04:36:11.207999 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:11.208002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:11.208069 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:11.208173 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:36:11.208588 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.209439 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.210770 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:11.211115 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:11.211209 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:11.211654 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:36:11.212227 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:36:11.212754 (+ 527us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:11.212791 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 04:36:11.212854 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 04:36:11 dist-test-slave-xt4k krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990970, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:11.224439 29966 raft_consensus.cc:1261] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 1 FOLLOWER]: Refusing update from remote peer 32ada4678d8d4e7fb0a2c1740223fbad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:36:11.224434 30163 raft_consensus.cc:1261] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 1 FOLLOWER]: Refusing update from remote peer 32ada4678d8d4e7fb0a2c1740223fbad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:36:11.225452 30289 consensus_queue.cc:1035] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a5ee58cd98a4dcc87461422fd606803" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:36:11.225688 30284 consensus_queue.cc:1035] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b87671f56f5b44fbbbdae745e7c52f5f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:36:11.233767 30293 mvcc.cc:205] Tried to move back new op lower bound from 7315419018124570624 to 7315419017951707136. Current Snapshot: MvccSnapshot[applied={T|T < 7315419018124570624}]
I20260806 04:36:11.236598 30294 mvcc.cc:205] Tried to move back new op lower bound from 7315419018124570624 to 7315419017951707136. Current Snapshot: MvccSnapshot[applied={T|T < 7315419018124570624}]
I20260806 04:36:11.248659 29510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:36:11.248852 29510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:36:11.255558 29731 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2c941d613d4a5b9defcdeb82137303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:36:11 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47858
I20260806 04:36:11.256623 30305 tablet_replica.cc:330] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad: stopping tablet replica
I20260806 04:36:11.256971 30305 raft_consensus.cc:2229] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:36:11.257076 30121 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2c941d613d4a5b9defcdeb82137303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:36:11 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45942
I20260806 04:36:11.257683 30305 raft_consensus.cc:2258] T bf2c941d613d4a5b9defcdeb82137303 P 32ada4678d8d4e7fb0a2c1740223fbad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:36:11.257891 29510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 596dca73c1ad4c178aae7ef79436483c: Sending DeleteTablet for 3 replicas of tablet bf2c941d613d4a5b9defcdeb82137303
I20260806 04:36:11.258388 30121 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2c941d613d4a5b9defcdeb82137303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:36:11 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:45942
I20260806 04:36:11.259060 29924 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2c941d613d4a5b9defcdeb82137303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:36:11 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:37822
I20260806 04:36:11.259825 29731 tablet_service.cc:1514] Processing DeleteTablet for tablet bf2c941d613d4a5b9defcdeb82137303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:36:11 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47858
I20260806 04:36:11.260934 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 29630
I20260806 04:36:11.267444 30307 tablet_replica.cc:330] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: stopping tablet replica
I20260806 04:36:11.267863 30307 raft_consensus.cc:2229] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:36:11.268229 30308 tablet_replica.cc:330] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: stopping tablet replica
I20260806 04:36:11.268708 30307 raft_consensus.cc:2258] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:36:11.269093 30308 raft_consensus.cc:2229] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:36:11.269661 30308 raft_consensus.cc:2258] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:36:11.271595 30307 ts_tablet_manager.cc:1905] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:36:11.273054 30308 ts_tablet_manager.cc:1905] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:36:11.275991 30307 ts_tablet_manager.cc:1918] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 04:36:11.277717 29493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:33239 (error 108)
I20260806 04:36:11.278791 30308 ts_tablet_manager.cc:1918] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:36:11.278997 30308 log.cc:1196] T bf2c941d613d4a5b9defcdeb82137303 P b87671f56f5b44fbbbdae745e7c52f5f: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/bf2c941d613d4a5b9defcdeb82137303
I20260806 04:36:11.279470 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 29839
I20260806 04:36:11.279546 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.279196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:33239 (local address 127.0.0.1:47872)
0806 04:36:11.279346 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.279428 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 04:36:11.291097 29493 catalog_manager.cc:4584] TS 32ada4678d8d4e7fb0a2c1740223fbad (127.12.15.193:33239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf2c941d613d4a5b9defcdeb82137303: Network error: Client connection negotiation failed: client connection to 127.12.15.193:33239: connect: Connection refused (error 111)
I20260806 04:36:11.313144 30307 log.cc:1196] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/bf2c941d613d4a5b9defcdeb82137303
I20260806 04:36:11.314750 30307 ts_tablet_manager.cc:1939] T bf2c941d613d4a5b9defcdeb82137303 P 0a5ee58cd98a4dcc87461422fd606803: Deleting consensus metadata
W20260806 04:36:11.316627 29496 catalog_manager.cc:4808] TS 0a5ee58cd98a4dcc87461422fd606803 (127.12.15.195:38493): delete failed for tablet bf2c941d613d4a5b9defcdeb82137303 because the tablet was not found. No further retry: Not found: Tablet not found: bf2c941d613d4a5b9defcdeb82137303
I20260806 04:36:11.322093 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 30043
I20260806 04:36:11.322281 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:11.321987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:46139 (local address 127.0.0.1:37836)
0806 04:36:11.322124 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:11.322185 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.194:46139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 04:36:11.341864 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 29464
I20260806 04:36:11.366122 12351 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 04:36:11.714373 12351 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 04:36:11.714546 12351 mini_ranger.cc:66] Stopping Ranger...
I20260806 04:36:12.059793 12351 mini_ranger.cc:68] Stopped Ranger
2026-08-06 04:36:12.060 UTC [28837] LOG: received smart shutdown request
2026-08-06 04:36:12.064 UTC [28837] LOG: background worker "logical replication launcher" (PID 28850) exited with exit code 1
2026-08-06 04:36:12.064 UTC [28841] LOG: shutting down
2026-08-06 04:36:12.217 UTC [28837] LOG: database system is shut down
2026-08-06T04:36:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48309 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:36:12 dist-test-slave-xt4k krb5kdc[30321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:36:12 dist-test-slave-xt4k krb5kdc[30321](info): set up 2 sockets
Aug 06 04:36:12 dist-test-slave-xt4k krb5kdc[30321](info): commencing operation
krb5kdc: starting...
W20260806 04:36:13.463574 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:36:13 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785990973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:36:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:36:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/rangeradmin_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/rangeradmin_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/rangerlookup_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/rangerlookup_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
I20260806 04:36:13.633221 12351 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5_1OQt/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/postgres -l logfile start
2026-08-06 04:36:16.126 UTC [30370] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 04:36:16.126 UTC [30370] LOG: listening on IPv4 address "127.12.15.212", port 43493
2026-08-06 04:36:16.129 UTC [30370] LOG: listening on Unix socket "/tmp/.s.PGSQL.43493"
2026-08-06 04:36:16.157 UTC [30378] LOG: database system was shut down at 2026-08-06 04:36:14 UTC
2026-08-06 04:36:16.161 UTC [30370] LOG: database system is ready to accept connections
I20260806 04:36:17.220284 12351 mini_postgres.cc:96] Postgres bound to 43493
2026-08-06 04:36:17.227 UTC [30389] FATAL: database "slave" does not exist
127.12.15.212:43493 - accepting connections
I20260806 04:36:17.227902 12351 mini_ranger.cc:161] Starting Ranger...
I20260806 04:36:17.243307 12351 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 04:36:17.701972 12351 mini_ranger.cc:87] Created ranger Postgres database
I20260806 04:36:17.702059 12351 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-admin
2026-08-06 04:36:17,978 [I] DB FLAVOR :POSTGRES
2026-08-06 04:36:17,978 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:36:17,978 [I] Checking connection..
2026-08-06 04:36:17,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 04:36:18,257 [I] Checking connection passed.
2026-08-06 04:36:18,257 [I] --------- Verifying version history table ---------
2026-08-06 04:36:18,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:36:18,495 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:36:18,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:36:18,749 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 04:36:18,749 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 04:36:18,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 04:36:19.004 UTC [30486] WARNING: there is no transaction in progress
2026-08-06 04:36:19,013 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 04:36:19,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 04:36:19,270 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 04:36:19,271 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:36:19,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 04:36:19,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 04:36:19,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xt4k.c.gcp-upstream.internal','N') ;"
2026-08-06 04:36:20,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:36:20,314 [I] Table x_portal_user does not exist in database ranger
2026-08-06 04:36:20,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:36:20,567 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 04:36:20,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:36:20,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:36:21,081 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 04:36:21,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 04:36:22.053 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.069 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.086 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.102 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.117 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.134 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.188 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.194 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.208 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.217 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.227 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.241 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.250 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.256 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.263 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22.269 UTC [30651] WARNING: there is no transaction in progress
2026-08-06 04:36:22,876 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 04:36:22,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 04:36:23,131 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 04:36:23,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 04:36:23,370 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 04:36:23,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 04:36:23,636 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 04:36:23,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 04:36:23,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 04:36:24,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xt4k.c.gcp-upstream.internal';"
2026-08-06 04:36:24,391 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 04:36:24,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 04:36:24,651 [I] DB_PATCHES have already been applied
I20260806 04:36:24.656944 12351 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-admin:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5_1OQt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 04:36:24.657017 12351 mini_ranger.cc:193] Using host: 127.12.15.212
I20260806 04:36:24.661473 12351 mini_ranger.cc:227] Ranger admin URL: http://127.12.15.212:43243
Aug 06, 2026 4:36:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:36:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:36:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:36:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:36:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:36:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:36:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 04:36:25.912997 12351 mini_ranger.cc:160] Time spent starting Ranger: real 8.685s user 0.000s sys 0.011s
Aug 06, 2026 4:36:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:36:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 4:36:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:36:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:36:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 04:36:47.857537 12351 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/rangerkms_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/rangerkms_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.15.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/HTTP_127.12.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 04:36:47 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991007, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 04:36:47.947814 12351 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 04:36:47.957491 12351 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 04:36:48.886876 12351 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 04:36:48.886948 12351 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms
I20260806 04:36:48.886960 12351 mini_ranger_kms.cc:227] Using postgres at 127.12.15.212:43493
2026-08-06 04:36:49,050 [I] DB FLAVOR :POSTGRES
2026-08-06 04:36:49,050 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 04:36:49,051 [I] Checking connection
2026-08-06 04:36:49,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 04:36:49,289 [I] connection success
2026-08-06 04:36:49,289 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 04:36:49,289 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 04:36:49,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 04:36:49,531 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 04:36:49,531 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 04:36:49,531 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 04:36:49,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.212:43493/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 04:36:49,799 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 04:36:50.236161 12351 mini_ranger_kms.cc:310] Created kms service
I20260806 04:36:50.449584 12351 mini_ranger_kms.cc:326] Created kudu user
I20260806 04:36:50.490605 12351 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 04:36:50 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991007, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:50.915858 12351 mini_ranger_kms.cc:384] Added ranger policy
I20260806 04:36:50.916111 12351 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms:/tmp/dist-test-task5_1OQt/build/debug/bin/postgresql.jar:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5_1OQt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 04:36:50.916139 12351 mini_ranger_kms.cc:240] Using host: 127.12.15.212
I20260806 04:36:50.920714 12351 mini_ranger_kms.cc:276] Ranger KMS PID: 30918
I20260806 04:36:50.920795 12351 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.15.212:47513
Aug 06, 2026 4:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 04:36:53 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785991013, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:53.521819 12351 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.574s user 0.008s sys 0.141s
I20260806 04:36:53.521953 12351 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 04:36:53.522025 12351 mini_ranger_kms.cc:401] 127.12.15.212:47513/kms/v1/keys
Aug 06 04:36:53 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991013, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.212@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
Aug 06 04:36:54 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991013, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.212@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
Aug 06 04:36:56 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:36:56.419023 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36021
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36021
--ranger_config_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:56.537366 30991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:56.537658 30991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:56.537716 30991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:56.541810 30991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:36:56.541886 30991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:56.541913 30991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:56.541934 30991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:36:56.541960 30991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:36:56.547678 30991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.15.212:47513/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36021
--ranger_config_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:56.548851 30991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:56.549947 30991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:56.559425 31004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:56.559487 30991 server_base.cc:1034] running on GCE node
W20260806 04:36:56.559443 30999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:56.559443 31002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:56.560391 30991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:56.561574 30991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:56.562745 30991 hybrid_clock.cc:648] HybridClock initialized: now 1785991016562722 us; error 45 us; skew 500 ppm
Aug 06 04:36:56 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991016, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:56.570210 30991 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:36:56.571214 30991 webserver.cc:460] Webserver started at http://127.12.15.254:32961/ using document root <none> and password file <none>
I20260806 04:36:56.571761 30991 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:56.571818 30991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:56.572017 30991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:56 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991016, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:56.628505 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "e537576fdbd340a6be25d5a286822094"
format_stamp: "Formatted at 2026-08-06 04:36:56 on dist-test-slave-xt4k"
server_key: "6cf650bbcbbb06e9c57610f353e2a188"
server_key_iv: "dc5deafbb388a15214da7383b50644bf"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:56.629276 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "e537576fdbd340a6be25d5a286822094"
format_stamp: "Formatted at 2026-08-06 04:36:56 on dist-test-slave-xt4k"
server_key: "6cf650bbcbbb06e9c57610f353e2a188"
server_key_iv: "dc5deafbb388a15214da7383b50644bf"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:56.633157 30991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:36:56.650700 31014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:56.652412 30991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:36:56.652568 30991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "e537576fdbd340a6be25d5a286822094"
format_stamp: "Formatted at 2026-08-06 04:36:56 on dist-test-slave-xt4k"
server_key: "6cf650bbcbbb06e9c57610f353e2a188"
server_key_iv: "dc5deafbb388a15214da7383b50644bf"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:56.652719 30991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:56.680209 30991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:56.683282 30991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:56.683485 30991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:56.691022 30991 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36021
I20260806 04:36:56.691072 31099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36021 every 8 connection(s)
I20260806 04:36:56.692063 30991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:36:56.695271 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:36:56.699159 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 30991
I20260806 04:36:56.699291 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:36:56.699586 12351 external_mini_cluster.cc:1428] Setting key 46dc7a91e1912cc3ef5c3ad979c88ba2
I20260806 04:36:56.701633 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Bootstrap starting.
I20260806 04:36:56.704471 31100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Neither blocks nor log segments found. Creating new log.
I20260806 04:36:56.705359 31100 log.cc:824] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Log is configured to *not* fsync() on all Append() calls
I20260806 04:36:56.708037 31100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: No bootstrap required, opened a new log
Aug 06 04:36:56 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:56.711808 31100 raft_consensus.cc:348] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:56.712070 31100 raft_consensus.cc:374] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:36:56.712134 31100 raft_consensus.cc:729] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e537576fdbd340a6be25d5a286822094, State: Initialized, Role: FOLLOWER
I20260806 04:36:56.712652 31100 consensus_queue.cc:260] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:56.712782 31100 raft_consensus.cc:388] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:36:56.712842 31100 raft_consensus.cc:482] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:36:56.712939 31100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:36:56.713948 31100 raft_consensus.cc:504] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:56.714280 31100 leader_election.cc:302] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e537576fdbd340a6be25d5a286822094; no voters:
I20260806 04:36:56.714392 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:56.701089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.12.15.254:36021)
0806 04:36:56.701526 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:56.701542 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:56.701579 (+ 37us) server_negotiation.cc:407] Connection header received
0806 04:36:56.702247 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:56.702268 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:56.702641 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:56.702938 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:56.703910 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:56.705027 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:56.706104 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:56.706637 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:56.706759 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:56.709780 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:56.709786 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:56.709800 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:56.709829 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:56.711382 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:56.711873 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:56.711879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:56.711884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:56.712001 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:56.712301 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:56.712304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:56.712305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:56.712820 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:56.713044 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:56.713328 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 04:36:56.713625 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":113,"threads_started":1}
I20260806 04:36:56.714610 31100 leader_election.cc:288] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:36:56.714810 31107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:36:56.715009 31107 raft_consensus.cc:686] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 LEADER]: Becoming Leader. State: Replica: e537576fdbd340a6be25d5a286822094, State: Running, Role: LEADER
I20260806 04:36:56.715358 31107 consensus_queue.cc:237] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:56.715965 31100 sys_catalog.cc:564] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:36:56.717211 31108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e537576fdbd340a6be25d5a286822094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } } }
I20260806 04:36:56.717355 31108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: This master's current role is: LEADER
I20260806 04:36:56.717734 31109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e537576fdbd340a6be25d5a286822094. Latest consensus state: current_term: 1 leader_uuid: "e537576fdbd340a6be25d5a286822094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } } }
I20260806 04:36:56.717828 31109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: This master's current role is: LEADER
I20260806 04:36:56.718268 31100 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 04:36:56.718549 31121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:36:56.724377 31121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:36:56.730347 31121 catalog_manager.cc:1347] Generated new cluster ID: 3267c56707074be2b6f18d6b3179bd44
I20260806 04:36:56.730414 31121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:36:56.737282 31121 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:36:56.738267 31121 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:36:56.745137 31121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Generated new TSK 0
I20260806 04:36:56.745950 31121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:36:57.447595 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36021
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:57.571417 31164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:57.571875 31164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:57.571969 31164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:57.576865 31164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:57.576984 31164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:57.577121 31164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:36:57.583417 31164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.15.212:47513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:57.584887 31164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:57.585937 31164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:57.595381 31177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:57.595474 31183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:57.595538 31178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:57.595747 31164 server_base.cc:1034] running on GCE node
I20260806 04:36:57.596263 31164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:57.596900 31164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:57.598083 31164 hybrid_clock.cc:648] HybridClock initialized: now 1785991017598058 us; error 35 us; skew 500 ppm
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:57.600973 31164 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:36:57.602083 31164 webserver.cc:460] Webserver started at http://127.12.15.193:34035/ using document root <none> and password file <none>
I20260806 04:36:57.602869 31164 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:57.602972 31164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:57.603238 31164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:57.620240 31164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "410395c1933b44e8979766f252724345"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "5be8640f599878fb78161936bb86aa95"
server_key_iv: "2477a8670f0c62752347cd8911006ea0"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:57.621008 31164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "410395c1933b44e8979766f252724345"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "5be8640f599878fb78161936bb86aa95"
server_key_iv: "2477a8670f0c62752347cd8911006ea0"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:57.625065 31164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:36:57.640676 31192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:57.642419 31164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:36:57.642560 31164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "410395c1933b44e8979766f252724345"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "5be8640f599878fb78161936bb86aa95"
server_key_iv: "2477a8670f0c62752347cd8911006ea0"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:57.642694 31164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:57.681011 31164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:57.684739 31164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:57.684952 31164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:57.685703 31164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:57.686657 31164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:57.686708 31164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:57.686756 31164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:57.686785 31164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:57.696789 31164 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:33489
I20260806 04:36:57.696841 31359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:33489 every 8 connection(s)
I20260806 04:36:57.697899 31164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:36:57.698592 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 31164
I20260806 04:36:57.698717 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:36:57.698984 12351 external_mini_cluster.cc:1428] Setting key 71c24e2573b252d1523c331c91ac80bf
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:36:57.716212 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:57.700711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:60105 (local address 127.12.15.254:36021)
0806 04:36:57.703710 (+ 2999us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:57.703717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:57.703730 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:36:57.703763 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:57.703765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:57.703805 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:57.703891 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:57.705407 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.706293 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.708083 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.708469 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.709256 (+ 787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:57.711595 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:57.711599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:57.711602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:57.711631 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:57.713468 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:57.714135 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:57.714139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:57.714141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:57.714234 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:57.714663 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:57.714666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:57.714668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:57.714950 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:57.715060 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:57.715807 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 04:36:57.715952 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2926,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:36:57.717528 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:57.701203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36021 (local address 127.12.15.193:60105)
0806 04:36:57.701847 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:57.701910 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:57.703002 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:57.704114 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:57.704124 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:57.704506 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:57.705202 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:57.705223 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.706466 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.706469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:57.707902 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:57.707914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.708659 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.708664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:57.708873 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:57.709555 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:57.709577 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:57.711378 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:57.713604 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:57.713611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:57.713631 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:57.713988 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:57.714327 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:57.714331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:57.714334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:57.714537 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:57.715120 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:57.715128 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:57.715496 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 04:36:57.715728 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 04:36:57.716240 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":143,"threads_started":1}
I20260806 04:36:57.720361 31361 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36021
I20260806 04:36:57.720664 31361 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:57.721349 31361 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:57.723348 31047 ts_manager.cc:194] Registered new tserver with Master: 410395c1933b44e8979766f252724345 (127.12.15.193:33489)
I20260806 04:36:57.724960 31047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:60105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:36:57.737586 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36021
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 04:36:57.870111 31377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:57.870385 31377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:57.870457 31377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:57.874815 31377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:57.874892 31377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:57.874979 31377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:36:57.880741 31377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.15.212:47513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:57.881884 31377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:57.882982 31377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:57.894011 31390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:57.894273 31392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:57.894025 31389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:57.894649 31377 server_base.cc:1034] running on GCE node
I20260806 04:36:57.895156 31377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:57.896018 31377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:57.897307 31377 hybrid_clock.cc:648] HybridClock initialized: now 1785991017897275 us; error 40 us; skew 500 ppm
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:57.900942 31377 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:36:57.902206 31377 webserver.cc:460] Webserver started at http://127.12.15.194:39435/ using document root <none> and password file <none>
I20260806 04:36:57.902961 31377 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:57.903065 31377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:57.903313 31377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:57.921923 31377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "a454dbc01e3f48a39b00a9e39616ca10"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "8c90e7f2cdc93dedae90f788cc20dc1e"
server_key_iv: "9ddbc5a8c627a7956aabfe0ae398e93c"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:57.922948 31377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "a454dbc01e3f48a39b00a9e39616ca10"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "8c90e7f2cdc93dedae90f788cc20dc1e"
server_key_iv: "9ddbc5a8c627a7956aabfe0ae398e93c"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:57.929457 31377 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 04:36:57.947719 31400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:57.949556 31377 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:36:57.949704 31377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "a454dbc01e3f48a39b00a9e39616ca10"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "8c90e7f2cdc93dedae90f788cc20dc1e"
server_key_iv: "9ddbc5a8c627a7956aabfe0ae398e93c"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:57.949851 31377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:57.966070 31377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:57.970474 31377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:57.970661 31377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:57.971349 31377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:57.972307 31377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:57.972383 31377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:57.972446 31377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:57.972477 31377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:57.983362 31377 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:35773
I20260806 04:36:57.984406 31556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:35773 every 8 connection(s)
I20260806 04:36:57.984738 31377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:36:57.987075 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 31377
I20260806 04:36:57.987180 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:36:57.987561 12351 external_mini_cluster.cc:1428] Setting key a6bacdd8e7e317c784badda2e60af634
Aug 06 04:36:57 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:36:58.004630 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:57.991745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:49313 (local address 127.12.15.254:36021)
0806 04:36:57.991905 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:57.991910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:57.992091 (+ 181us) server_negotiation.cc:407] Connection header received
0806 04:36:57.993256 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:57.993260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:57.993321 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:57.993372 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:57.994782 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.995515 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.997198 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.997560 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.997642 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:58.000816 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:58.000821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:58.000824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:58.000854 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:58.002394 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:58.002956 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:58.002960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:58.002961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:58.003036 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:58.003364 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:58.003368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:58.003369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:58.003565 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:58.003656 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:58.004297 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 04:36:58.004461 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:36:58.005585 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:57.990613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36021 (local address 127.12.15.194:49313)
0806 04:36:57.991953 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:57.991998 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:57.993000 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:57.993809 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:57.993819 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:57.994132 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:57.994635 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:57.994648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.995666 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.995669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:57.997051 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:57.997063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:57.997863 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:57.997867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:57.998069 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:57.998776 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:57.998799 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:58.000606 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:58.002529 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:58.002536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:58.002585 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:58.002847 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:58.003121 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:58.003124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:58.003126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:58.003265 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:58.003713 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:58.003720 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:58.003971 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 04:36:58.004139 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 04:36:58.004511 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":141,"threads_started":1}
I20260806 04:36:58.008932 31557 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36021
I20260806 04:36:58.009219 31557 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:58.009850 31557 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:58.011011 31047 ts_manager.cc:194] Registered new tserver with Master: a454dbc01e3f48a39b00a9e39616ca10 (127.12.15.194:35773)
I20260806 04:36:58.011587 31047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:49313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:36:58.027312 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36021
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:58.173931 31562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:58.174257 31562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:58.174320 31562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:58.179455 31562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:58.179544 31562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:58.179637 31562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:36:58.185883 31562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.15.212:47513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:58.187229 31562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:58.188740 31562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:58.199556 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:58.199908 31583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:58.199517 31573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:58.200235 31562 server_base.cc:1034] running on GCE node
I20260806 04:36:58.200660 31562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:58.201306 31562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:58.202494 31562 hybrid_clock.cc:648] HybridClock initialized: now 1785991018202469 us; error 31 us; skew 500 ppm
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:58.205811 31562 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:36:58.206813 31562 webserver.cc:460] Webserver started at http://127.12.15.195:36767/ using document root <none> and password file <none>
I20260806 04:36:58.207538 31562 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:58.207633 31562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:36:58.207844 31562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:58.240026 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "732bc1a1d13a487384e552390b77d751"
format_stamp: "Formatted at 2026-08-06 04:36:58 on dist-test-slave-xt4k"
server_key: "947ccc0937500208d12c848988234da1"
server_key_iv: "5d89db054ffd6359223fce3a676234a5"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:58.241000 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "732bc1a1d13a487384e552390b77d751"
format_stamp: "Formatted at 2026-08-06 04:36:58 on dist-test-slave-xt4k"
server_key: "947ccc0937500208d12c848988234da1"
server_key_iv: "5d89db054ffd6359223fce3a676234a5"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:58.264582 31562 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.005s sys 0.000s
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991017, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:58.289520 31600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:58.291327 31562 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 04:36:58.291468 31562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "732bc1a1d13a487384e552390b77d751"
format_stamp: "Formatted at 2026-08-06 04:36:58 on dist-test-slave-xt4k"
server_key: "947ccc0937500208d12c848988234da1"
server_key_iv: "5d89db054ffd6359223fce3a676234a5"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:58.291613 31562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:58.360761 31562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:58.364372 31562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:58.364584 31562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:58.365335 31562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:58.366415 31562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:58.366492 31562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:58.366570 31562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:58.366602 31562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:58.377580 31562 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:42685
I20260806 04:36:58.377718 31782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:42685 every 8 connection(s)
I20260806 04:36:58.378571 31562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:36:58.382381 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 31562
I20260806 04:36:58.382525 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:36:58.382838 12351 external_mini_cluster.cc:1428] Setting key be56e6231d7a2822fb06aea3a209678b
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:58.393399 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:58.380638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:36913 (local address 127.12.15.254:36021)
0806 04:36:58.380776 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:58.380780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:58.381770 (+ 990us) server_negotiation.cc:407] Connection header received
0806 04:36:58.383019 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:58.383023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:58.383091 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:58.383148 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:58.384855 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:58.385498 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:58.386761 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:58.386993 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:58.387084 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:58.389831 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:58.389835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:58.389838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:58.389865 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:58.391192 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:58.391694 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:58.391698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:58.391700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:58.391835 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:58.392172 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:58.392175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:58.392176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:58.392410 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:58.392500 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:58.393093 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 04:36:58.393228 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:36:58.394296 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:58.381089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36021 (local address 127.12.15.195:36913)
0806 04:36:58.381615 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:58.381669 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:58.382762 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:58.383569 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:58.383579 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:58.383939 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:58.384693 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:58.384715 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:58.385632 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:58.385635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:58.386607 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:58.386628 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:58.387134 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:58.387138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:58.387316 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:58.388142 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:58.388160 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:58.389655 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:58.391307 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:58.391314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:58.391332 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:58.391599 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:58.391927 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:58.391931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:58.391933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:58.392080 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:58.392516 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:58.392524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:58.392836 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 04:36:58.393037 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 04:36:58.393360 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":137,"threads_started":1}
I20260806 04:36:58.395484 31783 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36021
I20260806 04:36:58.395737 31783 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:58.396330 31783 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:58.397562 31047 ts_manager.cc:194] Registered new tserver with Master: 732bc1a1d13a487384e552390b77d751 (127.12.15.195:42685)
I20260806 04:36:58.398260 31047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:36913
I20260806 04:36:58.398886 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:36:58.399492 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 31164
I20260806 04:36:58.413491 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 31377
I20260806 04:36:58.446029 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 31562
I20260806 04:36:58.458401 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 30991
I20260806 04:36:58.473414 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36021
--webserver_interface=127.12.15.254
--webserver_port=32961
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36021
--ranger_config_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:36:58.585999 31787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:58.586265 31787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:58.586302 31787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:58.590284 31787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:36:58.590340 31787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:58.590358 31787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:58.590372 31787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:36:58.590385 31787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:36:58.596297 31787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36021
--ranger_config_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=32961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:58.597337 31787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:58.598311 31787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:58.606629 31795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:58.606732 31787 server_base.cc:1034] running on GCE node
W20260806 04:36:58.606598 31794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:58.606608 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:58.607466 31787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:58.608537 31787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:58.609691 31787 hybrid_clock.cc:648] HybridClock initialized: now 1785991018609673 us; error 31 us; skew 500 ppm
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:58.612394 31787 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:36:58.613269 31787 webserver.cc:460] Webserver started at http://127.12.15.254:32961/ using document root <none> and password file <none>
I20260806 04:36:58.613780 31787 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:58.613848 31787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:58.694219 31787 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:36:58.696489 31813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:58.697780 31787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:36:58.697893 31787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "e537576fdbd340a6be25d5a286822094"
format_stamp: "Formatted at 2026-08-06 04:36:56 on dist-test-slave-xt4k"
server_key: "6cf650bbcbbb06e9c57610f353e2a188"
server_key_iv: "dc5deafbb388a15214da7383b50644bf"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:58.698444 31787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:58.711428 31787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:58.714190 31787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:58.714366 31787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:58.722054 31787 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36021
I20260806 04:36:58.722072 31876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36021 every 8 connection(s)
I20260806 04:36:58.723173 31787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:36:58.727550 31877 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:36:58.729292 31877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Bootstrap starting.
I20260806 04:36:58.731489 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 31787
I20260806 04:36:58.731963 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:33489
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:36021
--webserver_port=34035
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:36:58.749697 31877 log.cc:824] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Log is configured to *not* fsync() on all Append() calls
I20260806 04:36:58.758126 31877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:36:58.758555 31877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Bootstrap complete.
I20260806 04:36:58.764021 31877 raft_consensus.cc:348] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:58.764532 31877 raft_consensus.cc:729] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e537576fdbd340a6be25d5a286822094, State: Initialized, Role: FOLLOWER
I20260806 04:36:58.765132 31877 consensus_queue.cc:260] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:58.765280 31877 raft_consensus.cc:388] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:36:58.765368 31877 raft_consensus.cc:482] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:36:58.765467 31877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:36:58.767877 31877 raft_consensus.cc:504] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:58.768251 31877 leader_election.cc:302] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e537576fdbd340a6be25d5a286822094; no voters:
I20260806 04:36:58.768661 31877 leader_election.cc:288] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 04:36:58.768831 31883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 04:36:58.769150 31883 raft_consensus.cc:686] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [term 2 LEADER]: Becoming Leader. State: Replica: e537576fdbd340a6be25d5a286822094, State: Running, Role: LEADER
I20260806 04:36:58.769537 31883 consensus_queue.cc:237] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } }
I20260806 04:36:58.769959 31877 sys_catalog.cc:564] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:36:58.770644 31883 sys_catalog.cc:455] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e537576fdbd340a6be25d5a286822094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } } }
I20260806 04:36:58.770833 31883 sys_catalog.cc:458] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: This master's current role is: LEADER
I20260806 04:36:58.771126 31883 sys_catalog.cc:455] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e537576fdbd340a6be25d5a286822094. Latest consensus state: current_term: 2 leader_uuid: "e537576fdbd340a6be25d5a286822094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e537576fdbd340a6be25d5a286822094" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36021 } } }
I20260806 04:36:58.771265 31883 sys_catalog.cc:458] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094 [sys.catalog]: This master's current role is: LEADER
I20260806 04:36:58.771615 31899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:36:58.775058 31899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:36:58.775615 31877 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 04:36:58.775830 31899 catalog_manager.cc:1259] Loaded cluster ID: 3267c56707074be2b6f18d6b3179bd44
I20260806 04:36:58.775934 31899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:36:58.778959 31899 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:36:58.779666 31899 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Loaded TSK: 0
I20260806 04:36:58.781147 31899 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:36:58.905076 31880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:58.905561 31880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:58.905691 31880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:58.911803 31880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:58.912019 31880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:58.912182 31880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:36:58.920138 31880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:33489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=34035
--tserver_master_addrs=127.12.15.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:58.921876 31880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:58.923417 31880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:58.938566 31936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:58.939191 31935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:58.940209 31940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:58.940575 31880 server_base.cc:1034] running on GCE node
I20260806 04:36:58.941126 31880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:58.941937 31880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:58.943169 31880 hybrid_clock.cc:648] HybridClock initialized: now 1785991018943087 us; error 100 us; skew 500 ppm
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:58.946373 31880 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:36:58.947260 31880 webserver.cc:460] Webserver started at http://127.12.15.193:34035/ using document root <none> and password file <none>
I20260806 04:36:58.947978 31880 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:58.948137 31880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 04:36:58 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:58.985219 31880 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.004s sys 0.000s
I20260806 04:36:58.990875 31950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:58.992728 31880 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 04:36:58.992883 31880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "410395c1933b44e8979766f252724345"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "5be8640f599878fb78161936bb86aa95"
server_key_iv: "2477a8670f0c62752347cd8911006ea0"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:58.993667 31880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:59.035002 31880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:59.038805 31880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:59.039044 31880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:59.039772 31880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:59.040901 31880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:59.040990 31880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.041072 31880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:59.041118 31880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.050771 31880 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:33489
I20260806 04:36:59.052199 31880 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:36:59.055250 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 31880
I20260806 04:36:59.055732 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:35773
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:36021
--webserver_port=39435
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:36:59.056896 32116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:33489 every 8 connection(s)
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991018, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:59.088007 32121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.059205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36021 (local address 127.12.15.193:46385)
0806 04:36:59.062374 (+ 3169us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:59.062448 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:59.063528 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:59.065534 (+ 2006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:59.065544 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:59.065909 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:59.066579 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:59.066602 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.068620 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.068623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:59.069981 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:59.069994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.071250 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.071254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:59.071443 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.078431 (+ 6988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:59.078454 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:59.081554 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:59.084362 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:59.084370 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:59.084387 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:59.084757 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:59.085225 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:59.085229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:59.085230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:59.085419 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:59.086269 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:59.086276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:59.086590 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 04:36:59.086788 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 04:36:59.087141 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2956,"thread_start_us":211,"threads_started":1}
I20260806 04:36:59.089540 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.060929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:46385 (local address 127.12.15.254:36021)
0806 04:36:59.062059 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:59.062084 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:59.063840 (+ 1756us) server_negotiation.cc:407] Connection header received
0806 04:36:59.064136 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:59.064150 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:59.064587 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:59.064916 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:59.066797 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.068456 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.070140 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.070866 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.071019 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.081869 (+ 10850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:59.081889 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:59.081911 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:59.081950 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:59.084191 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.084912 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.084919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.084925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.085108 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.085537 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.085540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.085542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.086062 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:59.086210 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:59.088569 (+ 2359us) server_negotiation.cc:299] Negotiation successful
0806 04:36:59.088835 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":119,"threads_started":1}
I20260806 04:36:59.092145 32117 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36021
I20260806 04:36:59.092465 32117 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.093040 32117 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
W20260806 04:36:59.095300 32117 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:36021 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 04:36:59.095415 32117 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.095592 32117 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:59.096549 32117 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.096706 32117 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
W20260806 04:36:59.097468 32117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:36:59.232185 32120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:59.232589 32120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:59.232725 32120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:59.238842 32120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:59.238931 32120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:59.239032 32120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:36:59.247521 32120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:35773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=39435
--tserver_master_addrs=127.12.15.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:59.249140 32120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:59.250608 32120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:59.263255 32129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:59.263298 32130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:59.263258 32132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:59.263777 32120 server_base.cc:1034] running on GCE node
I20260806 04:36:59.264196 32120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:59.264911 32120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:59.266109 32120 hybrid_clock.cc:648] HybridClock initialized: now 1785991019266083 us; error 33 us; skew 500 ppm
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:59.269482 32120 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:36:59.270637 32120 webserver.cc:460] Webserver started at http://127.12.15.194:39435/ using document root <none> and password file <none>
I20260806 04:36:59.271250 32120 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:59.271343 32120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:59.292711 32120 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 04:36:59.295842 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.297260 32120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:36:59.297458 32120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "a454dbc01e3f48a39b00a9e39616ca10"
format_stamp: "Formatted at 2026-08-06 04:36:57 on dist-test-slave-xt4k"
server_key: "8c90e7f2cdc93dedae90f788cc20dc1e"
server_key_iv: "9ddbc5a8c627a7956aabfe0ae398e93c"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:59.298308 32120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:59.334144 32120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:59.337925 32120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:59.338173 32120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:59.338953 32120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:59.340082 32120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:59.340147 32120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.340202 32120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:59.340229 32120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.351325 32120 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:35773
I20260806 04:36:59.351363 32305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:35773 every 8 connection(s)
I20260806 04:36:59.352746 32120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:36:59.353529 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 32120
I20260806 04:36:59.354017 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.212:47513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:42685
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:36021
--webserver_port=36767
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:59.371870 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.356066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36021 (local address 127.12.15.194:36653)
0806 04:36:59.356854 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:59.356942 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:59.358039 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:59.358677 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:59.358686 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:59.359062 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:59.359712 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:59.359730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.360916 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.360920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:59.362321 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:59.362334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.363025 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.363028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:59.363179 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.363788 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:59.363810 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:59.366208 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:59.368306 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:59.368313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:59.368331 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:59.368734 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:59.369070 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:59.369074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:59.369075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:59.369268 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:59.369838 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:59.369845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:59.370183 (+ 338us) client_negotiation.cc:769] Sending connection context
0806 04:36:59.370396 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 04:36:59.370769 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":167,"threads_started":1}
I20260806 04:36:59.372648 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.356998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:36653 (local address 127.12.15.254:36021)
0806 04:36:59.357328 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:59.357333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:59.357353 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:36:59.358308 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:59.358312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:59.358371 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:59.358479 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:59.359895 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.360751 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.362493 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.362782 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.362846 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.366426 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:59.366431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:59.366433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:59.366463 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:59.368162 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.368855 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.368859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.368861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.368959 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.369388 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.369391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.369393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.369676 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:59.369776 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:59.372300 (+ 2524us) server_negotiation.cc:299] Negotiation successful
0806 04:36:59.372476 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260806 04:36:59.374506 32306 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36021
I20260806 04:36:59.374792 32306 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.375444 32306 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
W20260806 04:36:59.378949 32306 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:36021 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 04:36:59.379094 32306 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.379293 32306 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:59.380290 32306 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.380505 32306 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
W20260806 04:36:59.381538 32306 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:36:59.496966 32310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:36:59.497223 32310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:36:59.497267 32310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:36:59.501401 32310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:36:59.501466 32310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:36:59.501546 32310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:36:59.507419 32310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.212:47513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:42685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=36767
--tserver_master_addrs=127.12.15.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:36:59.508895 32310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:36:59.510236 32310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:36:59.522590 32323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:59.522552 32327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:36:59.522642 32322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:36:59.523663 32310 server_base.cc:1034] running on GCE node
I20260806 04:36:59.524125 32310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:36:59.524921 32310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:36:59.526121 32310 hybrid_clock.cc:648] HybridClock initialized: now 1785991019526070 us; error 50 us; skew 500 ppm
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:36:59.529822 32310 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:36:59.530951 32310 webserver.cc:460] Webserver started at http://127.12.15.195:36767/ using document root <none> and password file <none>
I20260806 04:36:59.531659 32310 fs_manager.cc:359] Metadata directory not provided
I20260806 04:36:59.531746 32310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:36:59.549427 32310 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:36:59.553162 32342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.554956 32310 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:36:59.555094 32310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "732bc1a1d13a487384e552390b77d751"
format_stamp: "Formatted at 2026-08-06 04:36:58 on dist-test-slave-xt4k"
server_key: "947ccc0937500208d12c848988234da1"
server_key_iv: "5d89db054ffd6359223fce3a676234a5"
server_key_version: "kuduclusterkey@0"
I20260806 04:36:59.555817 32310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:36:59.575421 32310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:36:59.578778 32310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:36:59.578965 32310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:36:59.579882 32310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:36:59.580862 32310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:36:59.580914 32310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.580968 32310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:36:59.581004 32310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:36:59.592134 32310 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:42685
I20260806 04:36:59.592430 32509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:42685 every 8 connection(s)
I20260806 04:36:59.593559 32310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:36:59.602464 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 32310
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:36:59.611966 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.595750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:44345 (local address 127.12.15.254:36021)
0806 04:36:59.595889 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:59.595894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:59.596986 (+ 1092us) server_negotiation.cc:407] Connection header received
0806 04:36:59.598327 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:59.598331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:59.598393 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:59.598488 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:59.599989 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.600829 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.603964 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.604387 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.604478 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.607649 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:59.607653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:59.607656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:59.607683 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:59.609352 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.610014 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.610018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.610019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.610115 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.610535 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.610548 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.610550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.610792 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:59.610889 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:59.611655 (+ 766us) server_negotiation.cc:299] Negotiation successful
0806 04:36:59.611789 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:36:59.613906 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.596209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36021 (local address 127.12.15.195:44345)
0806 04:36:59.596817 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 04:36:59.596874 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:36:59.598060 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:36:59.598694 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:36:59.598704 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:36:59.599082 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:36:59.599797 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:59.599819 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.600998 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.601002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:59.603809 (+ 2807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:36:59.603822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.604940 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.604944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:36:59.605172 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.605789 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:36:59.605811 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:36:59.607480 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:36:59.609486 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:59.609493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:59.609523 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:59.609888 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:59.610229 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:36:59.610232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:36:59.610234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:36:59.610420 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:36:59.610963 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:36:59.610970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:36:59.611327 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 04:36:59.612393 (+ 1066us) client_negotiation.cc:240] Negotiation successful
0806 04:36:59.612785 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":142,"threads_started":1}
I20260806 04:36:59.616024 32510 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36021
I20260806 04:36:59.616292 32510 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.616976 32510 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
W20260806 04:36:59.619268 32510 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:36021 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 04:36:59.619387 32510 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.619850 32510 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:59.620821 32510 heartbeater.cc:464] Registering TS with master...
I20260806 04:36:59.621013 32510 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:36:59.621083 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:36:59.603217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.12.15.254:36021)
0806 04:36:59.606011 (+ 2794us) server_negotiation.cc:206] Beginning negotiation
0806 04:36:59.606021 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:36:59.606034 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:36:59.606083 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:36:59.606086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:36:59.606133 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:36:59.606213 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:36:59.607052 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.607877 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.610559 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:36:59.610925 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:36:59.611024 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:36:59.616083 (+ 5059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:36:59.616088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:36:59.616091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:36:59.616117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:36:59.617902 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.618731 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.618735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.618737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.618834 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:36:59.620097 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:36:59.620101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:36:59.620102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:36:59.620295 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:36:59.620531 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 04:36:59.620757 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 04:36:59.620882 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2713,"thread_start_us":77,"threads_started":1}
W20260806 04:36:59.621968 32510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 04:36:59 dist-test-slave-xt4k krb5kdc[30321](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:00.097735 32117 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:00.098050 32117 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:37:00.099109 31834 ts_manager.cc:194] Registered new tserver with Master: 410395c1933b44e8979766f252724345 (127.12.15.193:33489)
I20260806 04:37:00.100499 31834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:46385
I20260806 04:37:00.381831 32306 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:00.382148 32306 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:37:00.383162 31834 ts_manager.cc:194] Registered new tserver with Master: a454dbc01e3f48a39b00a9e39616ca10 (127.12.15.194:35773)
I20260806 04:37:00.383889 31834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:36653
Aug 06 04:37:00 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:37:00.622241 32510 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:00.622584 32510 heartbeater.cc:511] Master 127.12.15.254:36021 requested a full tablet report, sending...
I20260806 04:37:00.623433 31834 ts_manager.cc:194] Registered new tserver with Master: 732bc1a1d13a487384e552390b77d751 (127.12.15.195:42685)
I20260806 04:37:00.624264 31834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:44345
I20260806 04:37:00.630827 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:37:00.646154 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.632855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46570 (local address 127.12.15.254:36021)
0806 04:37:00.633114 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:00.633119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:00.636631 (+ 3512us) server_negotiation.cc:407] Connection header received
0806 04:37:00.636874 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:00.636878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:00.636934 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:00.636998 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:00.637970 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.638793 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.639975 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.640377 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.640462 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.642304 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:00.642308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:00.642311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:00.642339 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:00.644060 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:00.644552 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:00.644556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:00.644558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:00.644644 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:00.644912 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:00.644915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:00.644917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:00.645136 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:00.645256 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:00.645851 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 04:37:00.645974 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260806 04:37:00.650537 31834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:37:00.655933 31834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 06 04:37:00 dist-test-slave-xt4k krb5kdc[30321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991019, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for HTTP/127.12.15.212@KRBTEST.COM
I20260806 04:37:00.683454 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.672575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:35773 (local address 127.0.0.1:43168)
0806 04:37:00.674630 (+ 2055us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:00.674684 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:00.674818 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:00.676228 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:00.676237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:00.676265 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:00.676437 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.676446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.679291 (+ 2845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.679294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.680675 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.680687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.682799 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.682803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.682929 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.682973 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 04:37:00.683084 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 04:37:00.683287 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1869,"thread_start_us":114,"threads_started":1}
I20260806 04:37:00.687318 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.672630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.12.15.194:35773)
0806 04:37:00.675408 (+ 2778us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:00.675415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:00.675441 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:37:00.675521 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:00.675526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:00.675866 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:00.676025 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:00.676586 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.677840 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.681539 (+ 3699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.682461 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.682569 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.682732 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:00.686989 (+ 4257us) server_negotiation.cc:299] Negotiation successful
0806 04:37:00.687138 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2666,"thread_start_us":60,"threads_started":1}
I20260806 04:37:00.689424 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.674027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:33489 (local address 127.0.0.1:47126)
0806 04:37:00.678083 (+ 4056us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:00.678096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:00.678181 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:00.685245 (+ 7064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:00.685248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:00.685265 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:00.685344 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.685349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.686769 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.686772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.687912 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.687924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.689016 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.689020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.689105 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.689150 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 04:37:00.689200 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 04:37:00.689271 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3990,"thread_start_us":97,"threads_started":1}
I20260806 04:37:00.689893 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.673739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42685 (local address 127.0.0.1:46238)
0806 04:37:00.675068 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:00.675080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:00.675164 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:00.679030 (+ 3866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:00.679033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:00.679052 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:00.679147 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.679154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.685558 (+ 6404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.685562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.686767 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.686779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.689413 (+ 2634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.689417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.689618 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.689642 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:37:00.689702 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 04:37:00.689770 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"thread_start_us":90,"threads_started":1}
I20260806 04:37:00.690030 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.674206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.12.15.193:33489)
0806 04:37:00.684561 (+ 10355us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:00.684568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:00.684594 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:37:00.684674 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:00.684680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:00.684895 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:00.685054 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:00.685472 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.686635 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.688050 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.688868 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.689611 (+ 743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.689759 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:00.689808 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 04:37:00.689899 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10223,"thread_start_us":118,"threads_started":1}
I20260806 04:37:00.690409 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.674135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.12.15.195:42685)
0806 04:37:00.678419 (+ 4284us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:00.678425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:00.678448 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:37:00.678522 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:00.678527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:00.678714 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:00.678848 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:00.680811 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.685388 (+ 4577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.688298 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.689098 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.689202 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:00.689348 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:00.690218 (+ 870us) server_negotiation.cc:299] Negotiation successful
0806 04:37:00.690303 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4175,"thread_start_us":61,"threads_started":1}
I20260806 04:37:00.690857 32185 tablet_service.cc:1467] Processing CreateTablet for tablet bf81afe7ace146caa586478a66f90097 (DEFAULT_TABLE table=test-table [id=5e7efec6e88244b7a0aa817aefdb53d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:00.692077 32185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf81afe7ace146caa586478a66f90097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:00.693365 32410 tablet_service.cc:1467] Processing CreateTablet for tablet bf81afe7ace146caa586478a66f90097 (DEFAULT_TABLE table=test-table [id=5e7efec6e88244b7a0aa817aefdb53d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:00.694439 32410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf81afe7ace146caa586478a66f90097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:00.699236 32018 tablet_service.cc:1467] Processing CreateTablet for tablet bf81afe7ace146caa586478a66f90097 (DEFAULT_TABLE table=test-table [id=5e7efec6e88244b7a0aa817aefdb53d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:00.700373 32018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf81afe7ace146caa586478a66f90097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:00.708972 32565 tablet_bootstrap.cc:492] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Bootstrap starting.
I20260806 04:37:00.711334 32565 tablet_bootstrap.cc:654] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:00.712198 32565 log.cc:824] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:00.712920 32566 tablet_bootstrap.cc:492] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: Bootstrap starting.
I20260806 04:37:00.715057 32566 tablet_bootstrap.cc:654] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:00.716001 32566 log.cc:824] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:00.718546 32565 tablet_bootstrap.cc:492] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: No bootstrap required, opened a new log
I20260806 04:37:00.718753 32565 ts_tablet_manager.cc:1397] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 04:37:00.719375 32567 tablet_bootstrap.cc:492] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Bootstrap starting.
I20260806 04:37:00.722211 32567 tablet_bootstrap.cc:654] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:00.722683 32565 raft_consensus.cc:348] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.722956 32565 raft_consensus.cc:374] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:00.723009 32565 raft_consensus.cc:729] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a454dbc01e3f48a39b00a9e39616ca10, State: Initialized, Role: FOLLOWER
I20260806 04:37:00.722998 32567 log.cc:824] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:00.723516 32565 consensus_queue.cc:260] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.725508 32566 tablet_bootstrap.cc:492] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: No bootstrap required, opened a new log
I20260806 04:37:00.725720 32566 ts_tablet_manager.cc:1397] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.004s
I20260806 04:37:00.729897 32566 raft_consensus.cc:348] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.730175 32566 raft_consensus.cc:374] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:00.730237 32566 raft_consensus.cc:729] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 410395c1933b44e8979766f252724345, State: Initialized, Role: FOLLOWER
I20260806 04:37:00.731465 32565 ts_tablet_manager.cc:1428] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Time spent starting tablet: real 0.013s user 0.001s sys 0.005s
I20260806 04:37:00.732012 32566 consensus_queue.cc:260] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.733038 32566 ts_tablet_manager.cc:1428] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 04:37:00.733395 32117 heartbeater.cc:503] Master 127.12.15.254:36021 was elected leader, sending a full tablet report...
I20260806 04:37:00.733475 32306 heartbeater.cc:503] Master 127.12.15.254:36021 was elected leader, sending a full tablet report...
I20260806 04:37:00.743052 32567 tablet_bootstrap.cc:492] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: No bootstrap required, opened a new log
I20260806 04:37:00.743302 32567 ts_tablet_manager.cc:1397] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260806 04:37:00.747561 32567 raft_consensus.cc:348] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.747855 32567 raft_consensus.cc:374] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:00.747912 32567 raft_consensus.cc:729] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732bc1a1d13a487384e552390b77d751, State: Initialized, Role: FOLLOWER
I20260806 04:37:00.748456 32567 consensus_queue.cc:260] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.749495 32567 ts_tablet_manager.cc:1428] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:37:00.749855 32510 heartbeater.cc:503] Master 127.12.15.254:36021 was elected leader, sending a full tablet report...
W20260806 04:37:00.813959 32118 tablet.cc:2369] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:37:00.845736 32512 tablet.cc:2369] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:37:00.867283 32308 tablet.cc:2369] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:37:00.994570 32574 raft_consensus.cc:482] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:00.994801 32574 raft_consensus.cc:504] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:00.995988 32574 leader_election.cc:288] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a454dbc01e3f48a39b00a9e39616ca10 (127.12.15.194:35773), 732bc1a1d13a487384e552390b77d751 (127.12.15.195:42685)
I20260806 04:37:01.000551 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.996510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:33975 (local address 127.12.15.195:42685)
0806 04:37:00.996748 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:00.996753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:00.996820 (+ 67us) server_negotiation.cc:407] Connection header received
0806 04:37:00.997070 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:00.997074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:00.997133 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:00.997225 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:00.997480 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.998095 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.999289 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.999958 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:01.000094 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:01.000154 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:01.000299 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 04:37:01.000408 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260806 04:37:01.000566 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.996286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42685 (local address 127.12.15.193:33975)
0806 04:37:00.996728 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:00.996744 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:00.996907 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:00.997234 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:00.997236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:00.997252 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:00.997372 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.997377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.998190 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.998193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.999150 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.999161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:01.000075 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:01.000079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:01.000174 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:01.000203 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 04:37:01.000273 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 04:37:01.000366 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":20,"thread_start_us":194,"threads_started":1}
I20260806 04:37:01.001389 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.996288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:35773 (local address 127.12.15.193:55019)
0806 04:37:00.996880 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:00.996892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:00.996979 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:00.997568 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:00.997570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:00.997590 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:00.997707 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.997713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.998899 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.998902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:00.999851 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:00.999862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:01.000962 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:01.000966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:01.001070 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:01.001095 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 04:37:01.001173 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 04:37:01.001239 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":18,"thread_start_us":137,"threads_started":1}
I20260806 04:37:01.001328 32444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf81afe7ace146caa586478a66f90097" candidate_uuid: "410395c1933b44e8979766f252724345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732bc1a1d13a487384e552390b77d751" is_pre_election: true
I20260806 04:37:01.001410 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:00.996748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:55019 (local address 127.12.15.194:35773)
0806 04:37:00.996883 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:00.996887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:00.996969 (+ 82us) server_negotiation.cc:407] Connection header received
0806 04:37:00.997371 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:00.997374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:00.997421 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:00.997559 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:00.997859 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:00.998714 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:00.999960 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:01.000520 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:01.000649 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:01.000698 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:01.001220 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 04:37:01.001296 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:37:01.001688 32444 raft_consensus.cc:2454] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 410395c1933b44e8979766f252724345 in term 0.
I20260806 04:37:01.002164 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf81afe7ace146caa586478a66f90097" candidate_uuid: "410395c1933b44e8979766f252724345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a454dbc01e3f48a39b00a9e39616ca10" is_pre_election: true
I20260806 04:37:01.002208 31955 leader_election.cc:302] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 410395c1933b44e8979766f252724345, 732bc1a1d13a487384e552390b77d751; no voters:
I20260806 04:37:01.002467 32227 raft_consensus.cc:2454] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 410395c1933b44e8979766f252724345 in term 0.
I20260806 04:37:01.002519 32574 raft_consensus.cc:2781] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:37:01.002621 32574 raft_consensus.cc:482] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:37:01.002676 32574 raft_consensus.cc:3037] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:01.004146 32574 raft_consensus.cc:504] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:01.004611 32574 leader_election.cc:288] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [CANDIDATE]: Term 1 election: Requested vote from peers a454dbc01e3f48a39b00a9e39616ca10 (127.12.15.194:35773), 732bc1a1d13a487384e552390b77d751 (127.12.15.195:42685)
I20260806 04:37:01.005010 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf81afe7ace146caa586478a66f90097" candidate_uuid: "410395c1933b44e8979766f252724345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a454dbc01e3f48a39b00a9e39616ca10"
I20260806 04:37:01.005142 32227 raft_consensus.cc:3037] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:01.006618 32227 raft_consensus.cc:2454] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 410395c1933b44e8979766f252724345 in term 1.
I20260806 04:37:01.006927 32444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf81afe7ace146caa586478a66f90097" candidate_uuid: "410395c1933b44e8979766f252724345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732bc1a1d13a487384e552390b77d751"
I20260806 04:37:01.007047 31951 leader_election.cc:302] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 410395c1933b44e8979766f252724345, a454dbc01e3f48a39b00a9e39616ca10; no voters:
I20260806 04:37:01.007056 32444 raft_consensus.cc:3037] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:01.007236 32574 raft_consensus.cc:2781] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:01.007453 32574 raft_consensus.cc:686] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [term 1 LEADER]: Becoming Leader. State: Replica: 410395c1933b44e8979766f252724345, State: Running, Role: LEADER
I20260806 04:37:01.007797 32574 consensus_queue.cc:237] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } }
I20260806 04:37:01.008275 32444 raft_consensus.cc:2454] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 410395c1933b44e8979766f252724345 in term 1.
I20260806 04:37:01.011588 31834 catalog_manager.cc:5495] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 reported cstate change: term changed from 0 to 1, leader changed from <none> to 410395c1933b44e8979766f252724345 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "410395c1933b44e8979766f252724345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "410395c1933b44e8979766f252724345" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 33489 } health_report { overall_health: HEALTHY } } }
I20260806 04:37:01.056854 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:01.053277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.12.15.193:33489)
0806 04:37:01.053399 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:01.053403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:01.053417 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:37:01.053605 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:01.053608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:01.053665 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:01.053753 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:01.053977 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:01.054479 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:01.055299 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:01.055516 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:01.055603 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:01.055877 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:01.056288 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:37:01.056415 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:01.056483 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 04:37:01.056542 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:37:01.062815 32444 raft_consensus.cc:1261] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 1 FOLLOWER]: Refusing update from remote peer 410395c1933b44e8979766f252724345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:01.062954 32227 raft_consensus.cc:1261] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 1 FOLLOWER]: Refusing update from remote peer 410395c1933b44e8979766f252724345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:01.063565 32574 consensus_queue.cc:1035] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "732bc1a1d13a487384e552390b77d751" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:01.063782 32596 consensus_queue.cc:1035] T bf81afe7ace146caa586478a66f90097 P 410395c1933b44e8979766f252724345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a454dbc01e3f48a39b00a9e39616ca10" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:01.077019 32603 mvcc.cc:205] Tried to move back new op lower bound from 7315419222265626624 to 7315419222052376576. Current Snapshot: MvccSnapshot[applied={T|T < 7315419222265626624}]
I20260806 04:37:01.082640 31834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:01.082796 31834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:01.085059 31834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e537576fdbd340a6be25d5a286822094: Sending DeleteTablet for 3 replicas of tablet bf81afe7ace146caa586478a66f90097
I20260806 04:37:01.085767 32185 tablet_service.cc:1514] Processing DeleteTablet for tablet bf81afe7ace146caa586478a66f90097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:01 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:43168
I20260806 04:37:01.085858 32018 tablet_service.cc:1514] Processing DeleteTablet for tablet bf81afe7ace146caa586478a66f90097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:01 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:47126
I20260806 04:37:01.086623 32619 tablet_replica.cc:330] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: stopping tablet replica
I20260806 04:37:01.086989 32619 raft_consensus.cc:2229] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:01.087069 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 31880
I20260806 04:37:01.087381 32619 raft_consensus.cc:2258] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:01.092814 32410 tablet_service.cc:1514] Processing DeleteTablet for tablet bf81afe7ace146caa586478a66f90097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:01 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:46238
I20260806 04:37:01.093050 32619 ts_tablet_manager.cc:1905] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:01.095491 32619 ts_tablet_manager.cc:1918] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:01.095585 32619 log.cc:1196] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/bf81afe7ace146caa586478a66f90097
I20260806 04:37:01.095894 32619 ts_tablet_manager.cc:1939] T bf81afe7ace146caa586478a66f90097 P a454dbc01e3f48a39b00a9e39616ca10: Deleting consensus metadata
I20260806 04:37:01.096129 32622 tablet_replica.cc:330] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: stopping tablet replica
I20260806 04:37:01.096587 32622 raft_consensus.cc:2229] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:01.096909 32622 raft_consensus.cc:2258] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:01.097167 31814 catalog_manager.cc:4841] TS a454dbc01e3f48a39b00a9e39616ca10 (127.12.15.194:35773): tablet bf81afe7ace146caa586478a66f90097 (table test-table [id=5e7efec6e88244b7a0aa817aefdb53d8]) successfully deleted
I20260806 04:37:01.098352 32622 ts_tablet_manager.cc:1905] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:01.100934 32622 ts_tablet_manager.cc:1918] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:01.101028 32622 log.cc:1196] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/bf81afe7ace146caa586478a66f90097
I20260806 04:37:01.101326 32622 ts_tablet_manager.cc:1939] T bf81afe7ace146caa586478a66f90097 P 732bc1a1d13a487384e552390b77d751: Deleting consensus metadata
I20260806 04:37:01.102517 31816 catalog_manager.cc:4841] TS 732bc1a1d13a487384e552390b77d751 (127.12.15.195:42685): tablet bf81afe7ace146caa586478a66f90097 (table test-table [id=5e7efec6e88244b7a0aa817aefdb53d8]) successfully deleted
W20260806 04:37:01.102908 31817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:33489 (error 108)
I20260806 04:37:01.103168 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 32120
I20260806 04:37:01.106022 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:01.105748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:33489 (local address 127.0.0.1:47134)
0806 04:37:01.105879 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:01.105948 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:33489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 04:37:01.106163 31817 catalog_manager.cc:4584] TS 410395c1933b44e8979766f252724345 (127.12.15.193:33489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf81afe7ace146caa586478a66f90097: Network error: Client connection negotiation failed: client connection to 127.12.15.193:33489: connect: Connection refused (error 111)
I20260806 04:37:01.114463 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 32310
I20260806 04:37:01.124598 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 31787
I20260806 04:37:01.139174 12351 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 04:37:01.484009 12351 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 04:37:01.484139 12351 mini_ranger.cc:66] Stopping Ranger...
I20260806 04:37:01.829195 12351 mini_ranger.cc:68] Stopped Ranger
2026-08-06 04:37:01.829 UTC [30370] LOG: received smart shutdown request
2026-08-06 04:37:01.833 UTC [30370] LOG: background worker "logical replication launcher" (PID 30387) exited with exit code 1
2026-08-06 04:37:01.833 UTC [30381] LOG: shutting down
2026-08-06 04:37:01.995 UTC [30370] LOG: database system is shut down
2026-08-06T04:37:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49779 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:37:02 dist-test-slave-xt4k krb5kdc[32637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:37:02 dist-test-slave-xt4k krb5kdc[32637](info): set up 2 sockets
Aug 06 04:37:02 dist-test-slave-xt4k krb5kdc[32637](info): commencing operation
krb5kdc: starting...
W20260806 04:37:03.241950 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:37:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:37:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:37:03.371383 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33017
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:33017,127.12.15.253:36711,127.12.15.252:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:03.482547 32655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:03.482784 32655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:03.482821 32655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:03.486670 32655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:03.486719 32655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:03.486737 32655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:03.486752 32655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:03.486764 32655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:03.492053 32655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:33017,127.12.15.253:36711,127.12.15.252:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:03.493000 32655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:03.493930 32655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:03.502183 32669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:03.502178 32667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:03.502216 32666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:03.502243 32655 server_base.cc:1034] running on GCE node
I20260806 04:37:03.502978 32655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:03.503988 32655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:03.505160 32655 hybrid_clock.cc:648] HybridClock initialized: now 1785991023505138 us; error 40 us; skew 500 ppm
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:03.507962 32655 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:03.508908 32655 webserver.cc:460] Webserver started at http://127.12.15.254:42551/ using document root <none> and password file <none>
I20260806 04:37:03.509454 32655 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:03.509521 32655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:03.509737 32655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:03.511716 32655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "84528359a88445de926a409473b9d180"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "c03ff9436a7f490b10c9937a584e34c6"
server_key_iv: "1181566a77a5331b8fdbf5005d1a9be9"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.512255 32655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "84528359a88445de926a409473b9d180"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "c03ff9436a7f490b10c9937a584e34c6"
server_key_iv: "1181566a77a5331b8fdbf5005d1a9be9"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.515908 32655 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:37:03.518411 32681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:03.519634 32655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:03.519833 32655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "84528359a88445de926a409473b9d180"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "c03ff9436a7f490b10c9937a584e34c6"
server_key_iv: "1181566a77a5331b8fdbf5005d1a9be9"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.519946 32655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:03.532500 32655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:03.535771 32655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:03.535961 32655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:03.542858 32655 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:33017
I20260806 04:37:03.542922 32760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:33017 every 8 connection(s)
I20260806 04:37:03.543800 32655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:03.546640 32761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:03.548403 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 32655
I20260806 04:37:03.548529 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:37:03.548799 12351 external_mini_cluster.cc:1428] Setting key ea15d369405563213ae3b95072641eec
I20260806 04:37:03.550526 32761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.15.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.253@KRBTEST.COM" created.
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:03.565245 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.552770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33017 (local address 127.0.0.1:41798)
0806 04:37:03.553326 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.553363 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:03.554194 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:03.554842 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:03.554847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:03.555277 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:03.555520 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.555533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.556665 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.556667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.557401 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.557412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.558098 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.558102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.558283 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.558798 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:03.558813 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:03.560470 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:03.562541 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.562550 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.562558 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.562770 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.563117 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.563119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.563121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.563214 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.563942 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:03.563948 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:03.564080 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 04:37:03.564223 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 04:37:03.564508 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":36,"spinlock_wait_cycles":28544,"thread_start_us":136,"threads_started":1}
I20260806 04:37:03.565255 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.552770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.12.15.254:33017)
0806 04:37:03.553188 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.553206 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.553569 (+ 363us) server_negotiation.cc:407] Connection header received
0806 04:37:03.554486 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.554490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.554665 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.554831 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.555660 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.556553 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.557516 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.557954 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.558103 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.560616 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:03.560622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:03.560632 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:03.560655 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:03.562409 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.562879 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.562886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.562887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.563026 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.563294 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.563297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.563299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.563826 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:03.563960 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:03.564327 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 04:37:03.564590 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260806 04:37:03.567633 32761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:37:03.568328 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.568080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34574)
0806 04:37:03.568202 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.568272 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 04:37:03.568554 32682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
I20260806 04:37:03.569344 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.569151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34580)
0806 04:37:03.569247 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.569290 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 04:37:03.569494 32761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:36711: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab.
Entry for principal kudu/127.12.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab.
I20260806 04:37:03.575340 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:36711
--webserver_interface=127.12.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab
--principal=kudu/127.12.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:33017,127.12.15.253:36711,127.12.15.252:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:03.618741 32761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } attempt: 1
I20260806 04:37:03.619681 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.619329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34594)
0806 04:37:03.619531 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.619597 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 04:37:03.620541 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.620261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34598)
0806 04:37:03.620409 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.620479 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 04:37:03.620676 32761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:36711: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
W20260806 04:37:03.687536 303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:03.687865 303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:03.687937 303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 04:37:03.688830 32761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } attempt: 2
I20260806 04:37:03.689658 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.689345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34604)
0806 04:37:03.689493 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.689576 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 04:37:03.690479 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.690239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34608)
0806 04:37:03.690347 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.690393 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 04:37:03.690596 32761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:36711: Network error: Client connection negotiation failed: client connection to 127.12.15.253:36711: connect: Connection refused (error 111)
W20260806 04:37:03.692291 303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:03.692366 303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:03.692406 303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:03.692438 303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:03.692468 303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:03.698091 303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:33017,127.12.15.253:36711,127.12.15.252:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab
--principal=kudu/127.12.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:36711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:03.699208 303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:03.700217 303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:03.708716 312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:03.708709 309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:03.708709 310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:03.708938 303 server_base.cc:1034] running on GCE node
I20260806 04:37:03.709569 303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:03.710595 303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:03.711767 303 hybrid_clock.cc:648] HybridClock initialized: now 1785991023711738 us; error 41 us; skew 500 ppm
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:03.714649 303 init.cc:377] Logged in from keytab as kudu/127.12.15.253@KRBTEST.COM (short username kudu)
I20260806 04:37:03.715560 303 webserver.cc:460] Webserver started at http://127.12.15.253:43927/ using document root <none> and password file <none>
I20260806 04:37:03.716094 303 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:03.716169 303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:03.716419 303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:03.718552 303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data/instance:
uuid: "b76c989fee984c6989cf3f07f36ba4a3"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "f028f68ec2864b1e322aa08756aacaed"
server_key_iv: "585bb73db1a6e2aca1ab23ec0dba9027"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.719116 303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal/instance:
uuid: "b76c989fee984c6989cf3f07f36ba4a3"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "f028f68ec2864b1e322aa08756aacaed"
server_key_iv: "585bb73db1a6e2aca1ab23ec0dba9027"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.722870 303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 04:37:03.725263 323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:03.726516 303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:37:03.726645 303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal
uuid: "b76c989fee984c6989cf3f07f36ba4a3"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "f028f68ec2864b1e322aa08756aacaed"
server_key_iv: "585bb73db1a6e2aca1ab23ec0dba9027"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.726742 303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:03.747394 303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:03.750633 303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:03.750821 303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:03.757719 303 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.253:36711
I20260806 04:37:03.757736 409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.253:36711 every 8 connection(s)
I20260806 04:37:03.758699 303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
I20260806 04:37:03.761504 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 303
I20260806 04:37:03.761524 411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:03.761651 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-1/wal/instance
I20260806 04:37:03.761962 12351 external_mini_cluster.cc:1428] Setting key da02dca4e8ac613418008aad7c80e0c7
I20260806 04:37:03.765856 411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.15.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.252@KRBTEST.COM" created.
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:03.780589 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.767738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.12.15.254:33017)
0806 04:37:03.767890 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.767895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.768717 (+ 822us) server_negotiation.cc:407] Connection header received
0806 04:37:03.769651 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.769655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.769726 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.769842 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.771044 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.771589 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.773323 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.773681 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.773779 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.776687 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:03.776691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:03.776694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:03.776723 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:03.778324 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.778887 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.778890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.778892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.778987 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.779379 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.779382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.779384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.779625 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:03.779732 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:03.780247 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 04:37:03.780417 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:37:03.781289 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.768056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33017 (local address 127.0.0.1:41806)
0806 04:37:03.768551 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.768602 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:03.769420 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:03.770030 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:03.770037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:03.770390 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:03.770873 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.770887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.771735 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.771738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.772995 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.773027 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.773891 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.773897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.774115 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.774690 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:03.774716 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:03.776501 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:03.778454 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.778459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.778472 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.778768 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.779112 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.779114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.779116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.779274 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.779719 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:03.779724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:03.779979 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 04:37:03.780139 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 04:37:03.780438 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":119,"threads_started":1}
I20260806 04:37:03.781741 32761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } attempt: 3
I20260806 04:37:03.783092 411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab.
Entry for principal kudu/127.12.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab.
I20260806 04:37:03.791388 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:42119
--webserver_interface=127.12.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab
--principal=kudu/127.12.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:33017,127.12.15.253:36711,127.12.15.252:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:03.792431 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.783573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34620)
0806 04:37:03.783675 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.783686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:03.783756 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:03.784165 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:03.784168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:03.784298 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:03.784423 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.784431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.785194 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.785198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.785936 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.785946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.786478 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.786482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.786576 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.786939 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:03.786952 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:03.788843 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:03.790906 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.790909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.790912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.791087 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.791347 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.791350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.791352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.791469 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.791930 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:03.791933 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:03.792015 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 04:37:03.792112 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 04:37:03.792240 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":51,"spinlock_wait_cycles":1792}
I20260806 04:37:03.792546 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.782132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34618)
0806 04:37:03.782291 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.782302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:03.782357 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:03.782965 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:03.782968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:03.783140 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:03.783273 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.783280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.784456 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.784460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.785440 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.785451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.786254 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.786257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.786411 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.786847 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:03.786860 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:03.788295 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:03.790979 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.790983 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.790985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.791216 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.791579 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.791582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.791584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.791670 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.792043 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:03.792046 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:03.792148 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 04:37:03.792244 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 04:37:03.792399 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 04:37:03.792577 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.783707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.12.15.253:36711)
0806 04:37:03.783986 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.783992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.784002 (+ 10us) server_negotiation.cc:407] Connection header received
0806 04:37:03.784027 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.784029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.784058 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.784130 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.784520 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.785106 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.786051 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.786360 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.786452 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.788983 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:03.788987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:03.788989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:03.789015 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:03.790814 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.791189 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.791195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.791197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.791269 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.791565 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.791568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.791570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.791832 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:03.791951 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:03.792205 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 04:37:03.792372 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":124,"threads_started":1}
I20260806 04:37:03.793206 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.782224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.12.15.253:36711)
0806 04:37:03.782574 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.782586 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.782605 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:37:03.782641 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.782646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.782821 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.782949 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.783399 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.784331 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.785548 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.786146 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.786235 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.788439 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:03.788449 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:03.788465 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:03.788489 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:03.790854 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.791339 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.791342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.791344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.791450 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.791774 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.791776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.791778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.791960 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:03.792046 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:03.792941 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 04:37:03.793057 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":245,"thread_start_us":124,"threads_started":1}
I20260806 04:37:03.795194 411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:37:03.795220 32761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:37:03.795877 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.795654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35108)
0806 04:37:03.795766 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.795818 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 04:37:03.796182 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.795655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35118)
0806 04:37:03.796039 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.796123 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319}
W20260806 04:37:03.796434 326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
I20260806 04:37:03.796754 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.796493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35130)
0806 04:37:03.796660 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.796697 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260806 04:37:03.796911 32761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:42119: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
I20260806 04:37:03.797274 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.797111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35132)
0806 04:37:03.797190 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.797229 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 04:37:03.797392 411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:42119: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
I20260806 04:37:03.828059 32761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } attempt: 1
I20260806 04:37:03.828905 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.828616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35138)
0806 04:37:03.828761 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.828822 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 04:37:03.829622 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.829402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35150)
0806 04:37:03.829501 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.829544 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 04:37:03.829802 32761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:42119: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
I20260806 04:37:03.846634 411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } attempt: 1
I20260806 04:37:03.847584 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.847149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35154)
0806 04:37:03.847410 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.847483 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 04:37:03.848374 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.848134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35158)
0806 04:37:03.848241 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.848289 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 04:37:03.848557 411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:42119: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
I20260806 04:37:03.904999 32761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } attempt: 2
I20260806 04:37:03.905798 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.905504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35174)
0806 04:37:03.905648 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.905706 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 04:37:03.905732 427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:03.906001 427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:03.906095 427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 04:37:03.906472 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.906267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35188)
0806 04:37:03.906372 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.906416 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 04:37:03.906596 32761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:42119: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
W20260806 04:37:03.910108 427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:03.910156 427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:03.910181 427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:03.910203 427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:03.910233 427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:03.915285 427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:33017,127.12.15.253:36711,127.12.15.252:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab
--principal=kudu/127.12.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:42119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:03.916270 427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:03.916702 411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } attempt: 2
I20260806 04:37:03.917376 427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:37:03.917408 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.917131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35190)
0806 04:37:03.917282 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.917337 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 04:37:03.918155 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.917920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35202)
0806 04:37:03.918041 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.918087 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 04:37:03.918341 411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:42119: Network error: Client connection negotiation failed: client connection to 127.12.15.252:42119: connect: Connection refused (error 111)
W20260806 04:37:03.925827 442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:03.925863 427 server_base.cc:1034] running on GCE node
W20260806 04:37:03.925817 437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:03.925817 438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:03.926551 427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:03.927552 427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:03.928707 427 hybrid_clock.cc:648] HybridClock initialized: now 1785991023928695 us; error 31 us; skew 500 ppm
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:03.931324 427 init.cc:377] Logged in from keytab as kudu/127.12.15.252@KRBTEST.COM (short username kudu)
I20260806 04:37:03.932147 427 webserver.cc:460] Webserver started at http://127.12.15.252:35409/ using document root <none> and password file <none>
I20260806 04:37:03.932694 427 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:03.932761 427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:03.932964 427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:03.934984 427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data/instance:
uuid: "936ea82145bb4ac1a5d554004da59e45"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "8ada7b0ad2bce7d48daf8fbe4be858b3"
server_key_iv: "abbfef1f7c9080556a052fa3ed76ae89"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.935520 427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal/instance:
uuid: "936ea82145bb4ac1a5d554004da59e45"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "8ada7b0ad2bce7d48daf8fbe4be858b3"
server_key_iv: "abbfef1f7c9080556a052fa3ed76ae89"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.939070 427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 04:37:03.941426 450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:03.942565 427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:37:03.942678 427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal
uuid: "936ea82145bb4ac1a5d554004da59e45"
format_stamp: "Formatted at 2026-08-06 04:37:03 on dist-test-slave-xt4k"
server_key: "8ada7b0ad2bce7d48daf8fbe4be858b3"
server_key_iv: "abbfef1f7c9080556a052fa3ed76ae89"
server_key_version: "encryptionkey@0"
I20260806 04:37:03.942765 427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:03.968118 427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:03.970965 427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:03.971148 427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:03.977871 427 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.252:42119
I20260806 04:37:03.977936 531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.252:42119 every 8 connection(s)
I20260806 04:37:03.978785 427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
I20260806 04:37:03.979211 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 427
I20260806 04:37:03.979355 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785990859032933-12351-0/minicluster-data/master-2/wal/instance
I20260806 04:37:03.979621 12351 external_mini_cluster.cc:1428] Setting key a0f05120f896cdfea785a59461c27299
I20260806 04:37:03.982180 533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:03.986709 533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:37:03 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:03.994208 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.980925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.12.15.252:42119)
0806 04:37:03.981423 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.981441 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.981482 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:37:03.982310 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.982333 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.982757 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.983112 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.983753 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.985229 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.986048 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.986735 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.986894 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.989650 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:03.989655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:03.989664 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:03.989691 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:03.991407 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.991809 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.991815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.991821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.991939 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.992196 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.992199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.992201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.992663 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:03.992881 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:03.993140 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 04:37:03.993424 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"spinlock_wait_cycles":7552,"thread_start_us":120,"threads_started":1}
I20260806 04:37:03.997416 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.988642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.12.15.254:33017)
0806 04:37:03.988777 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.988782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.989057 (+ 275us) server_negotiation.cc:407] Connection header received
0806 04:37:03.989260 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.989264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.989320 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.989418 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.990102 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.990829 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.991697 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.991915 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.992016 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.994077 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:03.994081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:03.994083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:03.994108 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:03.995531 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.996110 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.996113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.996115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.996205 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:03.996507 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:03.996511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:03.996512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:03.996765 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:03.996854 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:03.997156 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 04:37:03.997265 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 04:37:03.997464 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.988560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:33017 (local address 127.0.0.1:41812)
0806 04:37:03.988934 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.988983 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:03.989145 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:03.989479 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:03.989484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:03.989842 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:03.989988 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.989997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.990946 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.990949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.991578 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.991588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.992017 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.992020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:03.992130 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:03.992622 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:03.992634 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:03.993943 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:03.995655 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.995662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.995667 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.995933 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.996298 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:03.996300 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:03.996301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:03.996418 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:03.996853 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:03.996859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:03.996971 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 04:37:03.997088 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 04:37:03.997267 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"spinlock_wait_cycles":4096,"thread_start_us":129,"threads_started":1}
I20260806 04:37:03.998759 533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:37:04 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
Aug 06 04:37:04 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
I20260806 04:37:04.005260 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.996538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.12.15.254:33017)
0806 04:37:03.996819 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.996825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.996837 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:37:03.996865 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.996867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.996901 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.996978 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:03.997565 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.998171 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.999044 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:03.999294 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:03.999399 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.002083 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.002087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.002089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.002114 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.003502 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.003938 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.003941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.003942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.004025 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.004377 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.004380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.004382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.004598 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.004801 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.004957 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.005068 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
I20260806 04:37:04.005738 32761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } attempt: 3
I20260806 04:37:04.007671 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.999318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34634 (local address 127.12.15.253:36711)
0806 04:37:03.999445 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:03.999451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:03.999463 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:37:03.999502 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:03.999504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:03.999541 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:03.999610 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.000029 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.000658 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.001527 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.001883 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.001993 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.003915 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.003919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.003922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.003950 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.005515 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.006230 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.006233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.006235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.006327 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.006752 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.006755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.006756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.006960 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.007042 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.007383 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.007525 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:37:04.008147 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:03.999195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:36711 (local address 127.0.0.1:34634)
0806 04:37:03.999331 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:03.999340 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:03.999407 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:03.999626 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:03.999629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:03.999785 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:03.999887 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:03.999894 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.000759 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.000763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.001388 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.001397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.001988 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.001991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.002086 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.002519 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:04.002531 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:04.003782 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:04.005609 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.005612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.005614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.005817 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.006473 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.006476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.006478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.006667 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.007086 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:04.007090 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:04.007186 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 04:37:04.007859 (+ 673us) client_negotiation.cc:240] Negotiation successful
0806 04:37:04.007993 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 04:37:04.009181 533 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:37:04.009486 411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } attempt: 3
Aug 06 04:37:04 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:37:04 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
I20260806 04:37:04.014707 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.006169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.12.15.252:42119)
0806 04:37:04.006265 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.006269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.006282 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:37:04.006344 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.006346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.006387 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.006452 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.006931 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.007415 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.008219 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.008519 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.008614 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.010658 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.010662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.010664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.010686 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.012977 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.013453 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.013456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.013458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.013536 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.013852 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.013855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.013857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.014086 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.014190 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.014452 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.014559 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 04:37:04.014707 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.006095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35228)
0806 04:37:04.006192 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:04.006205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:04.006273 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:04.006558 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:04.006561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:04.006733 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:04.006832 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.006839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.007507 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.007511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.008114 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.008124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.008721 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.008725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.008807 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.009163 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:04.009177 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:04.010519 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:04.013106 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.013110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.013112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.013353 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.013637 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.013640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.013642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.013731 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.014181 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:04.014184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:04.014286 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 04:37:04.014387 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 04:37:04.014525 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
Aug 06 04:37:04 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:37:04 dist-test-slave-xt4k krb5kdc[32637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991023, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
I20260806 04:37:04.017405 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.006485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.12.15.253:36711)
0806 04:37:04.006594 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.006598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.006676 (+ 78us) server_negotiation.cc:407] Connection header received
0806 04:37:04.006810 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.006813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.006855 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.006923 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.007674 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.008465 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.009387 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.009703 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.009798 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.012864 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.012868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.012870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.012896 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.014989 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.015465 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.015468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.015470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.015635 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.016218 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.016222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.016224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.016498 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.016755 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.017117 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.017242 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 04:37:04.019550 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: Bootstrap starting.
I20260806 04:37:04.020200 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.009602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35236)
0806 04:37:04.009778 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:04.009790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:04.009849 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:04.010942 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:04.010946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:04.011692 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:04.011771 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.011777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.012803 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.012807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.013525 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.013534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.014059 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.014063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.014135 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.015158 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:04.015172 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:04.016750 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:04.018742 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.018745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.018747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.018983 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.019252 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.019254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.019256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.019328 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.019754 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:04.019756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:04.019837 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 04:37:04.019908 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 04:37:04.020040 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 04:37:04.020422 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.010291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35236 (local address 127.12.15.252:42119)
0806 04:37:04.010628 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.010636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.010648 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:37:04.010672 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.010674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.010709 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.010982 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.011882 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.012689 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.013646 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.013953 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.014037 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.016873 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.016876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.016878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.016899 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.018635 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.019091 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.019094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.019095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.019170 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.019410 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.019413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.019414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.019659 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.020022 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.020097 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.020201 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":136,"threads_started":1}
I20260806 04:37:04.021301 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:37:04.022722 32761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:04.022783 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.009824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:42119 (local address 127.0.0.1:35250)
0806 04:37:04.009979 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:04.009992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:04.010064 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:04.011017 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:04.011020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:04.011189 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:04.011287 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.011294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.012901 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.012904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.014201 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:04.014212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.015079 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.015083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:04.015164 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.015547 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:04.015562 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:04.017574 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:04.021008 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.021011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.021013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.021270 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.021655 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:04.021658 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:04.021660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:04.021828 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:04.022278 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:04.022281 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:04.022372 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 04:37:04.022464 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 04:37:04.022607 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 04:37:04.023415 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.010510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.12.15.252:42119)
0806 04:37:04.010776 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.010785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.010799 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:37:04.010830 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.010833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.010872 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.011399 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.011520 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.012799 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.014596 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.014945 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.015344 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.018373 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.018376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.018379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.018406 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.020897 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.021467 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.021469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.021471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.021552 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.021962 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.021965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.021966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.022168 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.023072 (+ 904us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.023146 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.023255 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":65,"threads_started":1}
I20260806 04:37:04.023626 32761 log.cc:824] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:04.024042 533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: Bootstrap starting.
I20260806 04:37:04.025610 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: No bootstrap required, opened a new log
I20260806 04:37:04.026226 411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3: Bootstrap starting.
I20260806 04:37:04.026698 533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:04.027536 533 log.cc:824] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:04.028671 411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:04.028920 32761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.029323 32761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:04.029443 32761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84528359a88445de926a409473b9d180, State: Initialized, Role: FOLLOWER
I20260806 04:37:04.029600 411 log.cc:824] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:04.029734 533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: No bootstrap required, opened a new log
I20260806 04:37:04.030095 32761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.030597 557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.030743 557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.031483 32761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:04.034320 533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.034552 533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:04.034605 533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936ea82145bb4ac1a5d554004da59e45, State: Initialized, Role: FOLLOWER
I20260806 04:37:04.035039 533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.036324 533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 04:37:04.037545 32690 tablet.cc:2369] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:37:04.038621 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.028219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.12.15.254:33017)
0806 04:37:04.028360 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.028365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.029529 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 04:37:04.029769 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.029773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.029823 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.029877 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.030858 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.031389 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.032466 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.032802 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.032894 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.034613 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.034617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.034620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.034646 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.036310 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.037042 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.037046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.037048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.037136 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.037441 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.037445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.037447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.037668 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.037786 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.038146 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.038478 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 04:37:04.039706 411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3: No bootstrap required, opened a new log
I20260806 04:37:04.036316 32655 master_runner.cc:186] Error getting master registration for 127.12.15.253:36711: OK, instance_id { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" instance_seqno: 1785991023755738 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 04:37:04.040527 32655 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 04:37:04.043326 587 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:37:04.043396 587 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 04:37:04.043596 571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:37:04.043651 571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:37:04.043987 574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.044092 574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.044287 411 raft_consensus.cc:348] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.047515 411 raft_consensus.cc:374] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:04.047623 411 raft_consensus.cc:729] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b76c989fee984c6989cf3f07f36ba4a3, State: Initialized, Role: FOLLOWER
I20260806 04:37:04.048161 411 consensus_queue.cc:260] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.049292 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.024473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.12.15.253:36711)
0806 04:37:04.029265 (+ 4792us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.029270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.040917 (+ 11647us) server_negotiation.cc:407] Connection header received
0806 04:37:04.041093 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.041096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.041139 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.041184 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.041906 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.042652 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.043725 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.044036 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.044107 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.045771 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.045775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.045778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.045804 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.047403 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.047900 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.047903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.047905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.047999 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.048296 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.048299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.048300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.048558 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.048656 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.049028 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.049146 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4716}
I20260806 04:37:04.049624 411 sys_catalog.cc:564] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:04.049324 427 master_runner.cc:186] Error getting master registration for 127.12.15.253:36711: OK, instance_id { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" instance_seqno: 1785991023755738 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 04:37:04.052666 427 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 04:37:04.053517 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:04.028887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.12.15.252:42119)
0806 04:37:04.028989 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:04.028994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:04.032686 (+ 3692us) server_negotiation.cc:407] Connection header received
0806 04:37:04.032856 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:04.032859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:04.032906 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:04.032953 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:04.033870 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.034631 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.035702 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:04.036073 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:04.036154 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:04.037952 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:04.037956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:04.037958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:04.037982 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:04.046836 (+ 8854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.052102 (+ 5266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.052106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.052108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.052205 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:04.052559 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:04.052563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:04.052564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:04.052780 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:04.052886 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:04.053234 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 04:37:04.053364 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 04:37:04.057188 303 master_runner.cc:263] No leader master found from master b76c989fee984c6989cf3f07f36ba4a3
I20260806 04:37:04.057251 303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 04:37:04.057847 617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:37:04.057988 617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:37:04.062451 590 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.062661 590 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.062882 590 raft_consensus.cc:482] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:04.063069 590 raft_consensus.cc:504] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.064054 590 leader_election.cc:288] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84528359a88445de926a409473b9d180 (127.12.15.254:33017), 936ea82145bb4ac1a5d554004da59e45 (127.12.15.252:42119)
I20260806 04:37:04.064744 32720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b76c989fee984c6989cf3f07f36ba4a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84528359a88445de926a409473b9d180" is_pre_election: true
I20260806 04:37:04.065091 32720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b76c989fee984c6989cf3f07f36ba4a3 in term 0.
I20260806 04:37:04.065503 490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b76c989fee984c6989cf3f07f36ba4a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "936ea82145bb4ac1a5d554004da59e45" is_pre_election: true
I20260806 04:37:04.065805 490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b76c989fee984c6989cf3f07f36ba4a3 in term 0.
I20260806 04:37:04.066439 326 leader_election.cc:302] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 936ea82145bb4ac1a5d554004da59e45, b76c989fee984c6989cf3f07f36ba4a3; no voters:
I20260806 04:37:04.066735 618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:37:04.066823 618 raft_consensus.cc:482] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:37:04.066864 618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:04.068126 618 raft_consensus.cc:504] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.068532 618 leader_election.cc:288] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [CANDIDATE]: Term 1 election: Requested vote from peers 84528359a88445de926a409473b9d180 (127.12.15.254:33017), 936ea82145bb4ac1a5d554004da59e45 (127.12.15.252:42119)
I20260806 04:37:04.069258 490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b76c989fee984c6989cf3f07f36ba4a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "936ea82145bb4ac1a5d554004da59e45"
I20260806 04:37:04.069377 490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:04.071131 490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b76c989fee984c6989cf3f07f36ba4a3 in term 1.
I20260806 04:37:04.071444 32720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b76c989fee984c6989cf3f07f36ba4a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84528359a88445de926a409473b9d180"
I20260806 04:37:04.071586 32720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:04.072837 32720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b76c989fee984c6989cf3f07f36ba4a3 in term 1.
I20260806 04:37:04.073451 326 leader_election.cc:302] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84528359a88445de926a409473b9d180, b76c989fee984c6989cf3f07f36ba4a3; no voters:
I20260806 04:37:04.073634 618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:04.073745 618 raft_consensus.cc:686] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [term 1 LEADER]: Becoming Leader. State: Replica: b76c989fee984c6989cf3f07f36ba4a3, State: Running, Role: LEADER
I20260806 04:37:04.074077 618 consensus_queue.cc:237] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } }
I20260806 04:37:04.076571 590 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b76c989fee984c6989cf3f07f36ba4a3. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.076696 590 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:04.080617 621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:04.081646 621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:04.090389 490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [term 1 FOLLOWER]: Refusing update from remote peer b76c989fee984c6989cf3f07f36ba4a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:04.091182 618 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:04.095652 574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b76c989fee984c6989cf3f07f36ba4a3. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.095795 574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.096128 32720 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [term 1 FOLLOWER]: Refusing update from remote peer b76c989fee984c6989cf3f07f36ba4a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:04.096745 618 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:04.100653 557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b76c989fee984c6989cf3f07f36ba4a3. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.100796 557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.110378 590 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.110523 590 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:04.110905 624 mvcc.cc:205] Tried to move back new op lower bound from 7315419234654523392 to 7315419234610999296. Current Snapshot: MvccSnapshot[applied={T|T < 7315419234654523392}]
I20260806 04:37:04.110903 574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.111025 574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.112489 618 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.112610 618 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:04.113782 557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b76c989fee984c6989cf3f07f36ba4a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84528359a88445de926a409473b9d180" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33017 } } peers { permanent_uuid: "b76c989fee984c6989cf3f07f36ba4a3" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 36711 } } peers { permanent_uuid: "936ea82145bb4ac1a5d554004da59e45" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 42119 } } }
I20260806 04:37:04.113899 557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:37:04.116778 629 mvcc.cc:205] Tried to move back new op lower bound from 7315419234654523392 to 7315419234610999296. Current Snapshot: MvccSnapshot[applied={T|T < 7315419234654523392}]
I20260806 04:37:04.122056 621 catalog_manager.cc:1347] Generated new cluster ID: 6798537b91134d39a8b6228210cd1af4
I20260806 04:37:04.122123 621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:37:04.158265 621 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:37:04.159675 621 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:37:04.180173 621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b76c989fee984c6989cf3f07f36ba4a3: Generated new TSK 0
I20260806 04:37:04.181198 621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:37:05.044978 587 catalog_manager.cc:1259] Loaded cluster ID: 6798537b91134d39a8b6228210cd1af4
I20260806 04:37:05.044979 571 catalog_manager.cc:1259] Loaded cluster ID: 6798537b91134d39a8b6228210cd1af4
I20260806 04:37:05.045073 571 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: loading cluster ID for follower catalog manager: success
I20260806 04:37:05.045073 587 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: loading cluster ID for follower catalog manager: success
I20260806 04:37:05.047372 587 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: acquiring CA information for follower catalog manager: success
I20260806 04:37:05.047372 571 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: acquiring CA information for follower catalog manager: success
I20260806 04:37:05.048848 571 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84528359a88445de926a409473b9d180: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:37:05.048854 587 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 936ea82145bb4ac1a5d554004da59e45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:37:05.257488 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 32655
I20260806 04:37:05.269651 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 303
W20260806 04:37:05.281068 455 connection.cc:535] server connection from 127.0.0.1:35250 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 04:37:05.281365 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 427
2026-08-06T04:37:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3188 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:37:05 dist-test-slave-xt4k krb5kdc[647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:37:05 dist-test-slave-xt4k krb5kdc[647](info): set up 2 sockets
Aug 06 04:37:05 dist-test-slave-xt4k krb5kdc[647](info): commencing operation
krb5kdc: starting...
W20260806 04:37:06.427368 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:37:06 dist-test-slave-xt4k krb5kdc[647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:37:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:37:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:37:06.555235 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:37469
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:37469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:06.666031 669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:06.666287 669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:06.666326 669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:06.670203 669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:06.670252 669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:06.670270 669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:06.670285 669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:06.670300 669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:06.676297 669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:37469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:06.677407 669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:06.678396 669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:06.686544 679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:06.686887 677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:06.686969 676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:06.687153 669 server_base.cc:1034] running on GCE node
I20260806 04:37:06.687711 669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:06.688849 669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:06.690001 669 hybrid_clock.cc:648] HybridClock initialized: now 1785991026689981 us; error 37 us; skew 500 ppm
Aug 06 04:37:06 dist-test-slave-xt4k krb5kdc[647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991026, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:06.692673 669 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:06.693553 669 webserver.cc:460] Webserver started at http://127.12.15.254:37649/ using document root <none> and password file <none>
I20260806 04:37:06.694121 669 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:06.694191 669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:06.694394 669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:06.696447 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "9a155522b3d748eea8c91f9831677813"
format_stamp: "Formatted at 2026-08-06 04:37:06 on dist-test-slave-xt4k"
server_key: "2991e5369f161e2ed37287e6f8ca3136"
server_key_iv: "b22b874c41a170d648856f20a44e5b53"
server_key_version: "encryptionkey@0"
I20260806 04:37:06.697000 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "9a155522b3d748eea8c91f9831677813"
format_stamp: "Formatted at 2026-08-06 04:37:06 on dist-test-slave-xt4k"
server_key: "2991e5369f161e2ed37287e6f8ca3136"
server_key_iv: "b22b874c41a170d648856f20a44e5b53"
server_key_version: "encryptionkey@0"
I20260806 04:37:06.700608 669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:37:06.702914 688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:06.704062 669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:06.704175 669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "9a155522b3d748eea8c91f9831677813"
format_stamp: "Formatted at 2026-08-06 04:37:06 on dist-test-slave-xt4k"
server_key: "2991e5369f161e2ed37287e6f8ca3136"
server_key_iv: "b22b874c41a170d648856f20a44e5b53"
server_key_version: "encryptionkey@0"
I20260806 04:37:06.704267 669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:06.728977 669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:06.732075 669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:06.732259 669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:06.739252 669 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:37469
I20260806 04:37:06.739265 763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:37469 every 8 connection(s)
I20260806 04:37:06.740211 669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:06.742110 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 669
I20260806 04:37:06.742240 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:37:06.742573 12351 external_mini_cluster.cc:1428] Setting key 03bbcf1cb53c3404f958adccd2e01b1c
I20260806 04:37:06.743587 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:06.750083 764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813: Bootstrap starting.
Aug 06 04:37:06 dist-test-slave-xt4k krb5kdc[647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:06.752822 764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:06.753571 764 log.cc:824] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:06.755786 764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813: No bootstrap required, opened a new log
I20260806 04:37:06.756611 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:06.743868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57938 (local address 127.12.15.254:37469)
0806 04:37:06.744261 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:06.744276 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:06.744312 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:37:06.744984 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:06.745001 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:06.745325 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:06.745594 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:06.746494 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:06.747472 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:06.748617 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:06.749074 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:06.749191 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:06.751874 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:06.751881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:06.751897 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:06.751931 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:06.753801 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:06.754249 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:06.754256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:06.754262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:06.754404 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:06.754676 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:06.754677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:06.754679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:06.755066 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:06.755219 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:06.755495 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 04:37:06.755773 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260806 04:37:06.759654 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:37:06.759828 764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a155522b3d748eea8c91f9831677813" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 37469 } }
I20260806 04:37:06.760099 764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:06.760177 764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a155522b3d748eea8c91f9831677813, State: Initialized, Role: FOLLOWER
I20260806 04:37:06.760676 764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a155522b3d748eea8c91f9831677813" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 37469 } }
I20260806 04:37:06.760816 764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:37:06.760885 764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:37:06.760985 764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:06.761998 764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a155522b3d748eea8c91f9831677813" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 37469 } }
I20260806 04:37:06.762306 764 leader_election.cc:302] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a155522b3d748eea8c91f9831677813; no voters:
I20260806 04:37:06.762598 764 leader_election.cc:288] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:37:06.762778 770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:06.763057 770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [term 1 LEADER]: Becoming Leader. State: Replica: 9a155522b3d748eea8c91f9831677813, State: Running, Role: LEADER
I20260806 04:37:06.763411 770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a155522b3d748eea8c91f9831677813" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 37469 } }
I20260806 04:37:06.763950 764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:06.765242 771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a155522b3d748eea8c91f9831677813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a155522b3d748eea8c91f9831677813" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 37469 } } }
I20260806 04:37:06.765254 773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a155522b3d748eea8c91f9831677813. Latest consensus state: current_term: 1 leader_uuid: "9a155522b3d748eea8c91f9831677813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a155522b3d748eea8c91f9831677813" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 37469 } } }
I20260806 04:37:06.765430 773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:06.765697 771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:06.765813 790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:06.768813 790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:06.775162 790 catalog_manager.cc:1347] Generated new cluster ID: 4b91e8b68f5042469de70bea44bed5da
I20260806 04:37:06.775238 790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:37:06.789809 790 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:37:06.790782 790 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:37:06.798887 790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a155522b3d748eea8c91f9831677813: Generated new TSK 0
I20260806 04:37:06.799726 790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:37:06.815480 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 669
2026-08-06T04:37:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1540 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:37:06 dist-test-slave-xt4k krb5kdc[809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:37:06 dist-test-slave-xt4k krb5kdc[809](info): set up 2 sockets
Aug 06 04:37:06 dist-test-slave-xt4k krb5kdc[809](info): commencing operation
krb5kdc: starting...
W20260806 04:37:07.960071 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:37:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:37:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:37:08.092065 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38533
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:08.206149 831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:08.206435 831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:08.206504 831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:08.210582 831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:08.210646 831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:08.210673 831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:08.210696 831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:08.210724 831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:08.216048 831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:38533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:08.217203 831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:08.218251 831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:08.226974 838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:08.226958 837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.226974 831 server_base.cc:1034] running on GCE node
W20260806 04:37:08.226958 840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.227743 831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:08.228881 831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:08.230046 831 hybrid_clock.cc:648] HybridClock initialized: now 1785991028230019 us; error 37 us; skew 500 ppm
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:08.238301 831 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:08.239333 831 webserver.cc:460] Webserver started at http://127.12.15.254:33753/ using document root <none> and password file <none>
I20260806 04:37:08.239914 831 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:08.239979 831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:08.240175 831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:08.242329 831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "50f858fa3f954036b5b61a5c9ef552c4"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "852119160e3485eb4b20ae6caca648a5"
server_key_iv: "4106f9cee35b2c2fdb0e16079edbd67e"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.242926 831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "50f858fa3f954036b5b61a5c9ef552c4"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "852119160e3485eb4b20ae6caca648a5"
server_key_iv: "4106f9cee35b2c2fdb0e16079edbd67e"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.246794 831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 04:37:08.249320 852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.250679 831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:37:08.250818 831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "50f858fa3f954036b5b61a5c9ef552c4"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "852119160e3485eb4b20ae6caca648a5"
server_key_iv: "4106f9cee35b2c2fdb0e16079edbd67e"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.250919 831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:08.267741 831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:08.270890 831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:08.271083 831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:08.277900 831 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:38533
I20260806 04:37:08.278160 932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:38533 every 8 connection(s)
I20260806 04:37:08.278888 831 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:08.279304 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 831
I20260806 04:37:08.279438 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:37:08.279739 12351 external_mini_cluster.cc:1428] Setting key af0b333c241eafc1610a8446868c628f
I20260806 04:37:08.282114 935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:08.287725 935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap starting.
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:08.290138 935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:08.290853 935 log.cc:824] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:08.293098 935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: No bootstrap required, opened a new log
I20260806 04:37:08.293668 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:08.281151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.12.15.254:38533)
0806 04:37:08.281612 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:08.281629 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:08.281666 (+ 37us) server_negotiation.cc:407] Connection header received
0806 04:37:08.282356 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:08.282375 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:08.282727 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:08.283043 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:08.283847 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.284934 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.285770 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.286224 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.286374 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:08.288960 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:08.288966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:08.288978 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:08.289006 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:08.290607 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:08.291149 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:08.291155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:08.291160 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:08.291308 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:08.291672 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:08.291675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:08.291676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:08.292092 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:08.292244 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:08.292584 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 04:37:08.292869 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":142,"threads_started":1}
I20260806 04:37:08.296566 935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:08.296793 935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:08.296864 935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f858fa3f954036b5b61a5c9ef552c4, State: Initialized, Role: FOLLOWER
I20260806 04:37:08.297338 935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:08.297559 935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:37:08.297650 935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:37:08.297752 935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:08.298629 935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:08.298952 935 leader_election.cc:302] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50f858fa3f954036b5b61a5c9ef552c4; no voters:
I20260806 04:37:08.299270 935 leader_election.cc:288] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:37:08.299403 945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:08.299655 945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 LEADER]: Becoming Leader. State: Replica: 50f858fa3f954036b5b61a5c9ef552c4, State: Running, Role: LEADER
I20260806 04:37:08.299959 945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:08.300490 935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:08.301555 950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f858fa3f954036b5b61a5c9ef552c4. Latest consensus state: current_term: 1 leader_uuid: "50f858fa3f954036b5b61a5c9ef552c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } } }
I20260806 04:37:08.301683 950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:08.302083 962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:08.302021 947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50f858fa3f954036b5b61a5c9ef552c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } } }
I20260806 04:37:08.302189 947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:08.305438 962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:08.311877 962 catalog_manager.cc:1347] Generated new cluster ID: 2380d2fb99c34d4daa0256e7d3b346d2
I20260806 04:37:08.311949 962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:37:08.323571 962 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:37:08.324841 962 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:37:08.332881 962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Generated new TSK 0
I20260806 04:37:08.333699 962 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:37:08.368623 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38533
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:08.508421 979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:08.508674 979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:08.508718 979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:08.512703 979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:08.512759 979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:08.512827 979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:37:08.518042 979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:08.519043 979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:08.520033 979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:08.529588 986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:08.529614 987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.529709 979 server_base.cc:1034] running on GCE node
W20260806 04:37:08.529649 991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.530191 979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:08.530833 979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:08.532009 979 hybrid_clock.cc:648] HybridClock initialized: now 1785991028531978 us; error 37 us; skew 500 ppm
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:08.534750 979 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:37:08.535662 979 webserver.cc:460] Webserver started at http://127.12.15.193:38883/ using document root <none> and password file <none>
I20260806 04:37:08.536212 979 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:08.536275 979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:08.536484 979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:08.538489 979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "da2029162826452f8b73701183c04480"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "bb603a478a4b6031d1692217410a426d"
server_key_iv: "4fcd3f86630e31504532eed8192692cd"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.539011 979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "da2029162826452f8b73701183c04480"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "bb603a478a4b6031d1692217410a426d"
server_key_iv: "4fcd3f86630e31504532eed8192692cd"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.542761 979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:37:08.545312 1004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.546696 979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:37:08.546806 979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "da2029162826452f8b73701183c04480"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "bb603a478a4b6031d1692217410a426d"
server_key_iv: "4fcd3f86630e31504532eed8192692cd"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.546919 979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:08.558845 979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:08.561882 979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:08.562065 979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:08.562688 979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:08.563535 979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:08.563581 979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.563625 979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:08.563655 979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.572062 979 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:37881
I20260806 04:37:08.572099 1176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:37881 every 8 connection(s)
I20260806 04:37:08.572996 979 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:37:08.576630 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 979
I20260806 04:37:08.576754 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:37:08.577039 12351 external_mini_cluster.cc:1428] Setting key 914a106da0614a1bfb43083d6b206847
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:37:08.587824 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:08.574776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:36925 (local address 127.12.15.254:38533)
0806 04:37:08.574909 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:08.574913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:08.575719 (+ 806us) server_negotiation.cc:407] Connection header received
0806 04:37:08.577335 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:08.577339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:08.577399 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:08.577478 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:08.579033 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.579585 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.581082 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.581451 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.581556 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:08.584306 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:08.584309 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:08.584311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:08.584361 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:08.585659 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:08.586147 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:08.586151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:08.586152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:08.586230 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:08.586549 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:08.586553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:08.586555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:08.586848 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:08.586965 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:08.587500 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 04:37:08.587638 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:37:08.588773 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:08.575126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:36925)
0806 04:37:08.575569 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:08.575611 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:08.577108 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:08.577659 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:08.577667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:08.578151 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:08.578862 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:08.578882 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.579771 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.579780 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:08.580940 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:08.580952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.581629 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.581633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:08.581824 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:08.582434 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:08.582454 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:08.584123 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:08.585773 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:08.585779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:08.585795 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:08.586052 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:08.586316 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:08.586319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:08.586321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:08.586461 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:08.586976 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:08.586983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:08.587263 (+ 280us) client_negotiation.cc:769] Sending connection context
0806 04:37:08.587440 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 04:37:08.587788 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260806 04:37:08.590061 1177 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:37:08.590338 1177 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:08.590960 1177 heartbeater.cc:511] Master 127.12.15.254:38533 requested a full tablet report, sending...
I20260806 04:37:08.593169 883 ts_manager.cc:194] Registered new tserver with Master: da2029162826452f8b73701183c04480 (127.12.15.193:37881)
I20260806 04:37:08.594830 883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:36925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:37:08.607511 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38533
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:08.723680 1187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:08.723943 1187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:08.723985 1187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:08.728179 1187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:08.728238 1187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:08.728302 1187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:37:08.733814 1187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:08.734887 1187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:08.735930 1187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:08.745589 1192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:08.745589 1197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:08.745711 1194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.745743 1187 server_base.cc:1034] running on GCE node
I20260806 04:37:08.746362 1187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:08.747010 1187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:08.748207 1187 hybrid_clock.cc:648] HybridClock initialized: now 1785991028748175 us; error 42 us; skew 500 ppm
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:08.751000 1187 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:37:08.751873 1187 webserver.cc:460] Webserver started at http://127.12.15.194:39803/ using document root <none> and password file <none>
I20260806 04:37:08.752466 1187 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:08.752534 1187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:08.752708 1187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:08.754870 1187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "114816b98ffc490b92e6defd6a479584"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "aae052d0502da18137e60b62ce0b9b90"
server_key_iv: "32b8f45adef47cfbe4816d21fc879364"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.755398 1187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "114816b98ffc490b92e6defd6a479584"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "aae052d0502da18137e60b62ce0b9b90"
server_key_iv: "32b8f45adef47cfbe4816d21fc879364"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.759173 1187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 04:37:08.761641 1214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.762953 1187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:37:08.763078 1187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "114816b98ffc490b92e6defd6a479584"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "aae052d0502da18137e60b62ce0b9b90"
server_key_iv: "32b8f45adef47cfbe4816d21fc879364"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.763190 1187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:08.780823 1187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:08.783588 1187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:08.783759 1187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:08.784325 1187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:08.785210 1187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:08.785255 1187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.785300 1187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:08.785328 1187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.793509 1187 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:34805
I20260806 04:37:08.793533 1378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:34805 every 8 connection(s)
I20260806 04:37:08.794317 1187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:37:08.795197 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 1187
I20260806 04:37:08.795315 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:37:08.795634 12351 external_mini_cluster.cc:1428] Setting key 80ca78fa7a078bab1dcc2148e421b1ba
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:37:08.808995 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:08.796233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:46661 (local address 127.12.15.254:38533)
0806 04:37:08.796451 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:08.796458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:08.797421 (+ 963us) server_negotiation.cc:407] Connection header received
0806 04:37:08.799178 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:08.799182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:08.799257 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:08.799326 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:08.800931 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.801695 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.803049 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.803277 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.803363 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:08.805684 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:08.805688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:08.805691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:08.805717 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:08.806976 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:08.807470 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:08.807473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:08.807474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:08.807549 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:08.807864 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:08.807867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:08.807868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:08.808073 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:08.808170 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:08.808698 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 04:37:08.808821 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:37:08.809952 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:08.796663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:46661)
0806 04:37:08.797209 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:08.797292 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:08.798946 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:08.799553 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:08.799562 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:08.800015 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:08.800757 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:08.800776 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.801827 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.801831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:08.802927 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:08.802945 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:08.803423 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:08.803427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:08.803613 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:08.804145 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:08.804160 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:08.805531 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:08.807084 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:08.807090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:08.807104 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:08.807360 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:08.807628 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:08.807632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:08.807633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:08.807767 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:08.808166 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:08.808172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:08.808460 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 04:37:08.808618 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 04:37:08.808920 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260806 04:37:08.811096 1380 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38533
I20260806 04:37:08.811316 1380 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:08.811774 1380 heartbeater.cc:511] Master 127.12.15.254:38533 requested a full tablet report, sending...
I20260806 04:37:08.812817 883 ts_manager.cc:194] Registered new tserver with Master: 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805)
I20260806 04:37:08.813370 883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:46661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:37:08.826794 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38533
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:08.936162 1389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:08.936475 1389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:08.936537 1389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:08.940538 1389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:08.940603 1389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:08.940682 1389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:37:08.946156 1389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:08.947201 1389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:08.948196 1389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:08.957767 1399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.957851 1389 server_base.cc:1034] running on GCE node
W20260806 04:37:08.957762 1405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:08.957767 1401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:08.958461 1389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:08.959079 1389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:08.960249 1389 hybrid_clock.cc:648] HybridClock initialized: now 1785991028960229 us; error 29 us; skew 500 ppm
Aug 06 04:37:08 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:08.962987 1389 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:37:08.963837 1389 webserver.cc:460] Webserver started at http://127.12.15.195:46215/ using document root <none> and password file <none>
I20260806 04:37:08.964395 1389 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:08.964459 1389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:08.964653 1389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:08.966745 1389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "5fe3f77e533640989a3637c0162e3290"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "418f5297023bb94a9673f81ce385e198"
server_key_iv: "a61f848bf35ef8f8c000f45d57791511"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.967293 1389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "5fe3f77e533640989a3637c0162e3290"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "418f5297023bb94a9673f81ce385e198"
server_key_iv: "a61f848bf35ef8f8c000f45d57791511"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.970958 1389 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:37:08.973412 1414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:08.974690 1389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:37:08.974812 1389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5fe3f77e533640989a3637c0162e3290"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "418f5297023bb94a9673f81ce385e198"
server_key_iv: "a61f848bf35ef8f8c000f45d57791511"
server_key_version: "encryptionkey@0"
I20260806 04:37:08.974915 1389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:09.002637 1389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:09.005812 1389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:09.006016 1389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:09.006664 1389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:09.007546 1389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:09.007603 1389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.007650 1389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:09.007679 1389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.016412 1389 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39219
I20260806 04:37:09.016439 1590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39219 every 8 connection(s)
I20260806 04:37:09.017333 1389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:37:09.023418 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 1389
I20260806 04:37:09.023543 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:37:09.023947 12351 external_mini_cluster.cc:1428] Setting key 6ba578bd28119360bc59d236c9afcbb2
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991028, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:09.032248 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.019213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:43825 (local address 127.12.15.254:38533)
0806 04:37:09.019368 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.019372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.020227 (+ 855us) server_negotiation.cc:407] Connection header received
0806 04:37:09.021645 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.021648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.021701 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.021788 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:09.023201 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.023916 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.025328 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.025605 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.025702 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.028582 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:09.028585 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:09.028588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:09.028615 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:09.030010 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.030518 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.030521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.030522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.030623 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.031032 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.031035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.031036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.031249 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:09.031327 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.031988 (+ 661us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.032118 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 04:37:09.033228 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.019562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:43825)
0806 04:37:09.020063 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.020109 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.021419 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.021997 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.022004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.022379 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:09.023033 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.023052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.024055 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.024058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.025193 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.025207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.025774 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.025778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.025990 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.026577 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:09.026594 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:09.028375 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:09.030134 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.030141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.030155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.030430 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.030741 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.030744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.030745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.030920 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.031354 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:09.031360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:09.031644 (+ 284us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.031821 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.032214 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":114,"threads_started":1}
I20260806 04:37:09.034472 1591 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38533
I20260806 04:37:09.034739 1591 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:09.035256 1591 heartbeater.cc:511] Master 127.12.15.254:38533 requested a full tablet report, sending...
I20260806 04:37:09.036387 883 ts_manager.cc:194] Registered new tserver with Master: 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
I20260806 04:37:09.037235 883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:43825
I20260806 04:37:09.038125 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:37:09.047739 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.040482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59378 (local address 127.12.15.254:38533)
0806 04:37:09.040621 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.040625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.040801 (+ 176us) server_negotiation.cc:407] Connection header received
0806 04:37:09.040890 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.040892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.040930 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.040994 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:09.041681 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.042196 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.043066 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.043283 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.043361 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.044754 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:09.044758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:09.044760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:09.044783 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:09.046122 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.046488 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.046491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.046492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.046608 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.046844 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.046847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.046848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.047071 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:09.047178 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.047451 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.047585 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 04:37:09.050547 883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:37:09.053613 883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:37:09.068646 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.063388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.12.15.195:39219)
0806 04:37:09.063807 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.063819 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.063846 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:37:09.063937 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.063943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.064149 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.064547 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:09.064859 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.065994 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.067335 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.067944 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.068044 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.068201 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.068378 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.068482 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":146,"threads_started":1}
I20260806 04:37:09.068666 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.062910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39219 (local address 127.0.0.1:48116)
0806 04:37:09.063684 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.063699 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.063807 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.064358 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.064365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.064405 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:09.064726 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.064737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.066179 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.066183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.067147 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.067156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.068059 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.068063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.068168 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.068205 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.068316 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.068483 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":167,"spinlock_wait_cycles":7040,"thread_start_us":99,"threads_started":1}
I20260806 04:37:09.068709 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.062689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:37881 (local address 127.0.0.1:60804)
0806 04:37:09.063056 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.063105 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.063228 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.064365 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.064369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.064403 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:09.064666 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.064676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.065772 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.065775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.067170 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.067181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.068274 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.068278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.068395 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.068421 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.068467 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.068536 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":153,"spinlock_wait_cycles":5760,"thread_start_us":97,"threads_started":1}
I20260806 04:37:09.069314 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.063423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34805 (local address 127.0.0.1:56520)
0806 04:37:09.064199 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.064212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.064296 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.065226 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.065229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.065249 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:09.065326 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.065332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.066744 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.066748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.067878 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.067889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.068968 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.068972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.069053 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.069087 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.069134 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.069198 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260806 04:37:09.069692 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.062851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.12.15.193:37881)
0806 04:37:09.063844 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.063860 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.063879 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:37:09.063940 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.063944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.064149 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.064518 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:09.064791 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.065659 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.067306 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.068143 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.068822 (+ 679us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.069272 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.069321 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.069523 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":108,"threads_started":1}
I20260806 04:37:09.069906 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.064031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.12.15.194:34805)
0806 04:37:09.064774 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.064785 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.064808 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:37:09.064878 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.064882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.065086 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.065233 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:09.065433 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.066479 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.068029 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.068843 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.069492 (+ 649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.069630 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.069678 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.069766 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":125,"threads_started":1}
I20260806 04:37:09.071875 1485 tablet_service.cc:1467] Processing CreateTablet for tablet ad29423c1002445f8b856d5042b261b8 (DEFAULT_TABLE table=test-table [id=1c4c3bd560834b3da481cab6d8e45754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:09.072945 1485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad29423c1002445f8b856d5042b261b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:09.072755 1077 tablet_service.cc:1467] Processing CreateTablet for tablet ad29423c1002445f8b856d5042b261b8 (DEFAULT_TABLE table=test-table [id=1c4c3bd560834b3da481cab6d8e45754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:09.072755 1277 tablet_service.cc:1467] Processing CreateTablet for tablet ad29423c1002445f8b856d5042b261b8 (DEFAULT_TABLE table=test-table [id=1c4c3bd560834b3da481cab6d8e45754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:09.073812 1277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad29423c1002445f8b856d5042b261b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:09.073812 1077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad29423c1002445f8b856d5042b261b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:09.080035 1615 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Bootstrap starting.
I20260806 04:37:09.081691 1618 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Bootstrap starting.
I20260806 04:37:09.082134 1620 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Bootstrap starting.
I20260806 04:37:09.082636 1615 tablet_bootstrap.cc:654] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:09.084033 1615 log.cc:824] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:09.084383 1618 tablet_bootstrap.cc:654] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:09.085034 1620 tablet_bootstrap.cc:654] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:09.085619 1618 log.cc:824] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:09.085884 1620 log.cc:824] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:09.087307 1615 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: No bootstrap required, opened a new log
I20260806 04:37:09.087554 1615 ts_tablet_manager.cc:1397] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 04:37:09.087824 1618 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: No bootstrap required, opened a new log
I20260806 04:37:09.087986 1620 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: No bootstrap required, opened a new log
I20260806 04:37:09.088027 1618 ts_tablet_manager.cc:1397] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 04:37:09.088176 1620 ts_tablet_manager.cc:1397] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:37:09.091095 1620 raft_consensus.cc:348] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.091332 1620 raft_consensus.cc:374] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:09.091384 1620 raft_consensus.cc:729] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114816b98ffc490b92e6defd6a479584, State: Initialized, Role: FOLLOWER
I20260806 04:37:09.091627 1618 raft_consensus.cc:348] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.091596 1615 raft_consensus.cc:348] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.091852 1615 raft_consensus.cc:374] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:09.091852 1618 raft_consensus.cc:374] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:09.091900 1618 raft_consensus.cc:729] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2029162826452f8b73701183c04480, State: Initialized, Role: FOLLOWER
I20260806 04:37:09.091900 1615 raft_consensus.cc:729] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3f77e533640989a3637c0162e3290, State: Initialized, Role: FOLLOWER
I20260806 04:37:09.091871 1620 consensus_queue.cc:260] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.092378 1618 consensus_queue.cc:260] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.092959 1620 ts_tablet_manager.cc:1428] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260806 04:37:09.093027 1380 heartbeater.cc:503] Master 127.12.15.254:38533 was elected leader, sending a full tablet report...
I20260806 04:37:09.092976 1615 consensus_queue.cc:260] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.093339 1618 ts_tablet_manager.cc:1428] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:37:09.093631 1591 heartbeater.cc:503] Master 127.12.15.254:38533 was elected leader, sending a full tablet report...
I20260806 04:37:09.093662 1177 heartbeater.cc:503] Master 127.12.15.254:38533 was elected leader, sending a full tablet report...
I20260806 04:37:09.093904 1615 ts_tablet_manager.cc:1428] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:37:09.119654 1627 raft_consensus.cc:482] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:09.119885 1627 raft_consensus.cc:504] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.121085 1627 leader_election.cc:288] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
I20260806 04:37:09.125452 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.121486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:53163 (local address 127.12.15.195:39219)
0806 04:37:09.121626 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.121631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.121856 (+ 225us) server_negotiation.cc:407] Connection header received
0806 04:37:09.121985 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.121989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.122038 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.122117 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:09.122416 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.123035 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.124194 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.124822 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.124921 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.124972 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.125254 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.125326 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 04:37:09.125478 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.121471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:35865 (local address 127.12.15.194:34805)
0806 04:37:09.121599 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.121603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.121998 (+ 395us) server_negotiation.cc:407] Connection header received
0806 04:37:09.122120 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.122123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.122176 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.122251 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:09.122516 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.122986 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.124165 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.124804 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.124917 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.124973 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.125256 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.125321 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:37:09.125487 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.121369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34805 (local address 127.12.15.193:35865)
0806 04:37:09.121897 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.121912 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.122017 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.122265 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.122267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.122281 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:09.122388 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.122394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.123105 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.123108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.124024 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.124035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.124950 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.124954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.125109 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.125153 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.125227 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.125316 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":103,"thread_start_us":121,"threads_started":1}
I20260806 04:37:09.125489 1632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.121367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39219 (local address 127.12.15.193:53163)
0806 04:37:09.121757 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.121768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.121869 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.122135 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.122137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.122153 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:09.122291 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.122298 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.123158 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.123161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.124045 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.124054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.124950 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.124954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.125088 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.125153 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.125231 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.125316 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":67,"thread_start_us":148,"threads_started":1}
I20260806 04:37:09.126259 1309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad29423c1002445f8b856d5042b261b8" candidate_uuid: "da2029162826452f8b73701183c04480" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114816b98ffc490b92e6defd6a479584" is_pre_election: true
I20260806 04:37:09.126318 1515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad29423c1002445f8b856d5042b261b8" candidate_uuid: "da2029162826452f8b73701183c04480" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fe3f77e533640989a3637c0162e3290" is_pre_election: true
I20260806 04:37:09.126569 1309 raft_consensus.cc:2454] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da2029162826452f8b73701183c04480 in term 0.
I20260806 04:37:09.126605 1515 raft_consensus.cc:2454] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da2029162826452f8b73701183c04480 in term 0.
I20260806 04:37:09.127032 1007 leader_election.cc:302] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114816b98ffc490b92e6defd6a479584, da2029162826452f8b73701183c04480; no voters:
I20260806 04:37:09.127312 1627 raft_consensus.cc:2781] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:37:09.127394 1627 raft_consensus.cc:482] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:37:09.127424 1627 raft_consensus.cc:3037] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:09.128429 1627 raft_consensus.cc:504] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.128750 1627 leader_election.cc:288] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 1 election: Requested vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
I20260806 04:37:09.129174 1309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad29423c1002445f8b856d5042b261b8" candidate_uuid: "da2029162826452f8b73701183c04480" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114816b98ffc490b92e6defd6a479584"
I20260806 04:37:09.129163 1515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad29423c1002445f8b856d5042b261b8" candidate_uuid: "da2029162826452f8b73701183c04480" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fe3f77e533640989a3637c0162e3290"
I20260806 04:37:09.129279 1309 raft_consensus.cc:3037] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:09.129295 1515 raft_consensus.cc:3037] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:09.130267 1309 raft_consensus.cc:2454] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da2029162826452f8b73701183c04480 in term 1.
I20260806 04:37:09.130352 1515 raft_consensus.cc:2454] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da2029162826452f8b73701183c04480 in term 1.
I20260806 04:37:09.130524 1007 leader_election.cc:302] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114816b98ffc490b92e6defd6a479584, da2029162826452f8b73701183c04480; no voters:
I20260806 04:37:09.130719 1627 raft_consensus.cc:2781] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:09.130946 1627 raft_consensus.cc:686] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 LEADER]: Becoming Leader. State: Replica: da2029162826452f8b73701183c04480, State: Running, Role: LEADER
I20260806 04:37:09.131295 1627 consensus_queue.cc:237] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.134455 879 catalog_manager.cc:5495] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 reported cstate change: term changed from 0 to 1, leader changed from <none> to da2029162826452f8b73701183c04480 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "da2029162826452f8b73701183c04480" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } health_report { overall_health: UNKNOWN } } }
I20260806 04:37:09.149114 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.145114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.12.15.193:37881)
0806 04:37:09.145264 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.145268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.145283 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:37:09.145364 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.145368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.145438 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.145547 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:09.145869 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.146376 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.147268 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.147498 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.147630 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.148240 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:09.148750 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:37:09.148850 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.148936 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.148997 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:37:09.156051 1515 raft_consensus.cc:1261] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Refusing update from remote peer da2029162826452f8b73701183c04480: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:09.156091 1309 raft_consensus.cc:1261] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Refusing update from remote peer da2029162826452f8b73701183c04480: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:09.156926 1634 consensus_queue.cc:1035] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:09.157126 1627 consensus_queue.cc:1035] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [LEADER]: Connected to new peer: Peer: permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:09.164865 1645 mvcc.cc:205] Tried to move back new op lower bound from 7315419255414669312 to 7315419255324340224. Current Snapshot: MvccSnapshot[applied={T|T < 7315419255414669312}]
I20260806 04:37:09.167136 1644 mvcc.cc:205] Tried to move back new op lower bound from 7315419255414669312 to 7315419255324340224. Current Snapshot: MvccSnapshot[applied={T|T < 7315419255414669312}]
I20260806 04:37:09.172946 883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:09.173125 883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:09.175511 883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Sending DeleteTablet for 3 replicas of tablet ad29423c1002445f8b856d5042b261b8
I20260806 04:37:09.176367 1277 tablet_service.cc:1514] Processing DeleteTablet for tablet ad29423c1002445f8b856d5042b261b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:09 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:56520
I20260806 04:37:09.176376 1077 tablet_service.cc:1514] Processing DeleteTablet for tablet ad29423c1002445f8b856d5042b261b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:09 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:60804
I20260806 04:37:09.176450 1485 tablet_service.cc:1514] Processing DeleteTablet for tablet ad29423c1002445f8b856d5042b261b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:09 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:48116
I20260806 04:37:09.176857 1653 tablet_replica.cc:330] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: stopping tablet replica
I20260806 04:37:09.177093 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 979
I20260806 04:37:09.177129 1653 raft_consensus.cc:2229] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:37:09.177467 1653 raft_consensus.cc:2258] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:09.180886 1655 tablet_replica.cc:330] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: stopping tablet replica
I20260806 04:37:09.181152 1655 raft_consensus.cc:2229] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:09.181402 1655 raft_consensus.cc:2258] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:09.184079 1655 ts_tablet_manager.cc:1905] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:09.186457 1655 ts_tablet_manager.cc:1918] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:09.186553 1655 log.cc:1196] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/ad29423c1002445f8b856d5042b261b8
I20260806 04:37:09.186846 1655 ts_tablet_manager.cc:1939] T ad29423c1002445f8b856d5042b261b8 P 114816b98ffc490b92e6defd6a479584: Deleting consensus metadata
I20260806 04:37:09.187968 853 catalog_manager.cc:4841] TS 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): tablet ad29423c1002445f8b856d5042b261b8 (table test-table [id=1c4c3bd560834b3da481cab6d8e45754]) successfully deleted
I20260806 04:37:09.188359 1657 tablet_replica.cc:330] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: stopping tablet replica
I20260806 04:37:09.188791 1657 raft_consensus.cc:2229] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:09.189137 1657 raft_consensus.cc:2258] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:09.191048 1657 ts_tablet_manager.cc:1905] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:09.193199 1657 ts_tablet_manager.cc:1918] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:09.193295 1657 log.cc:1196] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/ad29423c1002445f8b856d5042b261b8
I20260806 04:37:09.193552 1657 ts_tablet_manager.cc:1939] T ad29423c1002445f8b856d5042b261b8 P 5fe3f77e533640989a3637c0162e3290: Deleting consensus metadata
I20260806 04:37:09.194677 857 catalog_manager.cc:4841] TS 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): tablet ad29423c1002445f8b856d5042b261b8 (table test-table [id=1c4c3bd560834b3da481cab6d8e45754]) successfully deleted
W20260806 04:37:09.198938 853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:37881 (error 108)
I20260806 04:37:09.199213 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 1187
I20260806 04:37:09.202319 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.201774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:37881 (local address 127.0.0.1:60828)
0806 04:37:09.202152 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.202226 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
W20260806 04:37:09.203087 853 catalog_manager.cc:4584] TS da2029162826452f8b73701183c04480 (127.12.15.193:37881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad29423c1002445f8b856d5042b261b8: Network error: Client connection negotiation failed: client connection to 127.12.15.193:37881: connect: Connection refused (error 111)
I20260806 04:37:09.207297 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 1389
I20260806 04:37:09.224476 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 831
I20260806 04:37:09.238582 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38533
--webserver_interface=127.12.15.254
--webserver_port=33753
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:09.369587 1661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:09.369858 1661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:09.369899 1661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:09.373857 1661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:09.373924 1661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:09.373941 1661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:09.373957 1661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:09.373970 1661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:09.379321 1661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:38533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=33753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:09.380446 1661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:09.381474 1661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:09.395988 1669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:09.395996 1671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:09.396253 1667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:09.396528 1661 server_base.cc:1034] running on GCE node
I20260806 04:37:09.397187 1661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:09.398419 1661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:09.399602 1661 hybrid_clock.cc:648] HybridClock initialized: now 1785991029399587 us; error 34 us; skew 500 ppm
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991029, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:09.402568 1661 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:09.403577 1661 webserver.cc:460] Webserver started at http://127.12.15.254:33753/ using document root <none> and password file <none>
I20260806 04:37:09.404153 1661 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:09.404228 1661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:09.408274 1661 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:37:09.410584 1684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.411935 1661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:37:09.412089 1661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "50f858fa3f954036b5b61a5c9ef552c4"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "852119160e3485eb4b20ae6caca648a5"
server_key_iv: "4106f9cee35b2c2fdb0e16079edbd67e"
server_key_version: "encryptionkey@0"
I20260806 04:37:09.412922 1661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:09.427695 1661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:09.431298 1661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:09.431507 1661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:09.439499 1661 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:38533
I20260806 04:37:09.440034 1763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:38533 every 8 connection(s)
I20260806 04:37:09.440829 1661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:09.445500 1765 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:37:09.447275 1765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap starting.
I20260806 04:37:09.448583 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 1661
I20260806 04:37:09.449327 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:37881
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:38533
--webserver_port=38883
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:09.465252 1765 log.cc:824] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:09.475080 1765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:37:09.475461 1765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap complete.
I20260806 04:37:09.479795 1765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:09.480222 1765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f858fa3f954036b5b61a5c9ef552c4, State: Initialized, Role: FOLLOWER
I20260806 04:37:09.480749 1765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:09.480876 1765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:37:09.480954 1765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:37:09.481053 1765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:37:09.483019 1765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:09.483351 1765 leader_election.cc:302] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50f858fa3f954036b5b61a5c9ef552c4; no voters:
I20260806 04:37:09.483680 1765 leader_election.cc:288] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 04:37:09.483780 1773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 04:37:09.483969 1773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 LEADER]: Becoming Leader. State: Replica: 50f858fa3f954036b5b61a5c9ef552c4, State: Running, Role: LEADER
I20260806 04:37:09.484287 1773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:09.485034 1765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:09.485730 1774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "50f858fa3f954036b5b61a5c9ef552c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } } }
I20260806 04:37:09.485860 1774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:09.486182 1775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f858fa3f954036b5b61a5c9ef552c4. Latest consensus state: current_term: 2 leader_uuid: "50f858fa3f954036b5b61a5c9ef552c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } } }
I20260806 04:37:09.486325 1775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:09.486652 1784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:09.491643 1784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:09.492552 1784 catalog_manager.cc:1259] Loaded cluster ID: 2380d2fb99c34d4daa0256e7d3b346d2
I20260806 04:37:09.492620 1784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:37:09.494131 1784 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 04:37:09.498539 1784 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 04:37:09.504317 1784 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:37:09.505138 1784 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Loaded TSK: 0
I20260806 04:37:09.506349 1784 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:37:09.570555 1771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:09.570776 1771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:09.570816 1771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:09.574747 1771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:09.574795 1771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:09.574852 1771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:37:09.579809 1771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:37881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=38883
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:09.580751 1771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:09.581799 1771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:09.594589 1811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:09.594696 1817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:09.594590 1813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:09.596197 1771 server_base.cc:1034] running on GCE node
I20260806 04:37:09.596626 1771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:09.597232 1771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:09.598390 1771 hybrid_clock.cc:648] HybridClock initialized: now 1785991029598364 us; error 47 us; skew 500 ppm
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991029, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:09.601192 1771 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:37:09.602083 1771 webserver.cc:460] Webserver started at http://127.12.15.193:38883/ using document root <none> and password file <none>
I20260806 04:37:09.602641 1771 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:09.602870 1771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:09.606724 1771 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 04:37:09.608814 1827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.609995 1771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:09.610090 1771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "da2029162826452f8b73701183c04480"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "bb603a478a4b6031d1692217410a426d"
server_key_iv: "4fcd3f86630e31504532eed8192692cd"
server_key_version: "encryptionkey@0"
I20260806 04:37:09.610615 1771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:09.631304 1771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:09.633952 1771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:09.634121 1771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:09.634687 1771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:09.635731 1839 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 04:37:09.639811 1771 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 04:37:09.639870 1771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 04:37:09.639935 1771 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 04:37:09.641628 1771 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 04:37:09.641670 1771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 04:37:09.641935 1839 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Bootstrap starting.
I20260806 04:37:09.650873 1771 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:37881
I20260806 04:37:09.650943 1997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:37881 every 8 connection(s)
I20260806 04:37:09.651758 1771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:37:09.657500 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 1771
I20260806 04:37:09.657971 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:34805
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:38533
--webserver_port=39803
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:09.660786 1839 log.cc:824] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Log is configured to *not* fsync() on all Append() calls
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991029, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:09.669881 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.653870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:52567 (local address 127.12.15.254:38533)
0806 04:37:09.654257 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.654280 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.654864 (+ 584us) server_negotiation.cc:407] Connection header received
0806 04:37:09.655920 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.655935 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.656359 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.656732 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:09.657714 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.658753 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.659934 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.660628 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.660793 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.663702 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:09.663709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:09.663726 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:09.663763 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:09.666120 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.666765 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.666771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.666777 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.666930 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.667325 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.667329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.667331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.667852 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:09.668073 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.668710 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.669035 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":120,"threads_started":1}
I20260806 04:37:09.669934 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.653777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:52567)
0806 04:37:09.654720 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.654757 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.655476 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.656760 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.656767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.657099 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:09.657533 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.657545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.658901 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.658904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.659772 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.659785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.660771 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.660775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.660966 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.661568 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:09.661586 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:09.663435 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:09.666254 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.666261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.666275 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.666637 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.667041 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.667044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.667046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.667227 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.667993 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:09.668000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:09.668309 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.668568 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.668933 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":143,"threads_started":1}
I20260806 04:37:09.670274 1839 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:37:09.670684 1839 tablet_bootstrap.cc:492] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Bootstrap complete.
I20260806 04:37:09.671710 1839 ts_tablet_manager.cc:1397] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Time spent bootstrapping tablet: real 0.030s user 0.017s sys 0.009s
I20260806 04:37:09.672768 2000 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38533
I20260806 04:37:09.673009 2000 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:09.673529 2000 heartbeater.cc:511] Master 127.12.15.254:38533 requested a full tablet report, sending...
I20260806 04:37:09.674219 1839 raft_consensus.cc:348] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.674597 1839 raft_consensus.cc:729] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2029162826452f8b73701183c04480, State: Initialized, Role: FOLLOWER
I20260806 04:37:09.675061 1839 consensus_queue.cc:260] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } } peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } }
I20260806 04:37:09.675879 1839 ts_tablet_manager.cc:1428] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260806 04:37:09.676277 1715 ts_manager.cc:194] Registered new tserver with Master: da2029162826452f8b73701183c04480 (127.12.15.193:37881)
I20260806 04:37:09.680270 1715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:52567
I20260806 04:37:09.682987 2000 heartbeater.cc:503] Master 127.12.15.254:38533 was elected leader, sending a full tablet report...
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991029, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:37:09.691183 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.679578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:37881 (local address 127.0.0.1:60836)
0806 04:37:09.680288 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.680323 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.680536 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.681042 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.681048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.681461 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:09.681570 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.681577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.682613 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.682617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.684504 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.684515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.685626 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.685630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.685744 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.686262 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:09.686274 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:09.687837 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:09.689551 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.689561 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.689566 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.689782 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.690051 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.690055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.690057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.690137 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.690553 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:09.690559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:09.690678 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.690794 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.690988 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":127,"threads_started":1}
I20260806 04:37:09.691196 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.679675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.12.15.193:37881)
0806 04:37:09.680050 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.680061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.680652 (+ 591us) server_negotiation.cc:407] Connection header received
0806 04:37:09.680709 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.680713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.680862 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.680974 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:09.681686 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.682456 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.684651 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.685409 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.685521 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.687965 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:09.687974 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:09.687984 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:09.688006 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:09.689446 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.689886 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.689891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.689892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.689966 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.690214 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.690218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.690220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.690443 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:09.690572 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.690887 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.691050 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":282,"thread_start_us":87,"threads_started":1}
I20260806 04:37:09.692160 1903 tablet_service.cc:1514] Processing DeleteTablet for tablet ad29423c1002445f8b856d5042b261b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:09 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:60836
I20260806 04:37:09.693069 2019 tablet_replica.cc:330] stopping tablet replica
I20260806 04:37:09.693390 2019 raft_consensus.cc:2229] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:09.693650 2019 raft_consensus.cc:2258] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:09.694752 2019 ts_tablet_manager.cc:1905] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:09.699239 2019 ts_tablet_manager.cc:1918] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:09.699342 2019 log.cc:1196] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/ad29423c1002445f8b856d5042b261b8
I20260806 04:37:09.699607 2019 ts_tablet_manager.cc:1939] T ad29423c1002445f8b856d5042b261b8 P da2029162826452f8b73701183c04480: Deleting consensus metadata
I20260806 04:37:09.701640 1685 catalog_manager.cc:4841] TS da2029162826452f8b73701183c04480 (127.12.15.193:37881): tablet ad29423c1002445f8b856d5042b261b8 (table test-table [id=1c4c3bd560834b3da481cab6d8e45754]) successfully deleted
W20260806 04:37:09.781021 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:09.781284 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:09.781327 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:09.785372 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:09.785432 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:09.785512 2007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:37:09.790771 2007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:34805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=39803
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:09.791785 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:09.792809 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:09.802228 2027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:09.802241 2031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:09.802241 2026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:09.802527 2007 server_base.cc:1034] running on GCE node
I20260806 04:37:09.802899 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:09.803514 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:09.804667 2007 hybrid_clock.cc:648] HybridClock initialized: now 1785991029804638 us; error 38 us; skew 500 ppm
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991029, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:09.807286 2007 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:37:09.808116 2007 webserver.cc:460] Webserver started at http://127.12.15.194:39803/ using document root <none> and password file <none>
I20260806 04:37:09.808696 2007 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:09.808777 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:09.812534 2007 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 04:37:09.814730 2039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.816098 2007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:09.816249 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "114816b98ffc490b92e6defd6a479584"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "aae052d0502da18137e60b62ce0b9b90"
server_key_iv: "32b8f45adef47cfbe4816d21fc879364"
server_key_version: "encryptionkey@0"
I20260806 04:37:09.816864 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:09.844599 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:09.848007 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:09.848203 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:09.848842 2007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:09.849737 2007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:09.849786 2007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.849843 2007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:09.849870 2007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:09.858769 2007 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:34805
I20260806 04:37:09.859035 2218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:34805 every 8 connection(s)
I20260806 04:37:09.859845 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:37:09.866601 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 2007
I20260806 04:37:09.867019 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:39219
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:38533
--webserver_port=46215
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:37:09 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991029, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:09.874065 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.861825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:35449 (local address 127.12.15.254:38533)
0806 04:37:09.861981 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:09.861986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:09.862883 (+ 897us) server_negotiation.cc:407] Connection header received
0806 04:37:09.864268 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:09.864272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:09.864321 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:09.864423 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:09.865591 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.866128 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.867473 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.867728 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.867826 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.870486 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:09.870490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:09.870492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:09.870519 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:09.871997 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.872571 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.872574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.872576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.872654 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:09.872982 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:09.872986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:09.872988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:09.873167 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:09.873254 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:09.873753 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 04:37:09.873890 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:37:09.874858 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:09.862218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:35449)
0806 04:37:09.862731 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:09.862775 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:09.864057 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:09.864574 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:09.864581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:09.864949 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:09.865430 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.865443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.866258 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.866261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.867344 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:09.867354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:09.867878 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:09.867881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:09.868054 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:09.868670 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:09.868687 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:09.870311 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:09.872127 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.872133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.872149 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.872472 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.872748 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:09.872751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:09.872753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:09.872892 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:09.873267 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:09.873274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:09.873523 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 04:37:09.873692 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 04:37:09.874028 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":115,"threads_started":1}
I20260806 04:37:09.876020 2219 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38533
I20260806 04:37:09.876251 2219 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:09.876760 2219 heartbeater.cc:511] Master 127.12.15.254:38533 requested a full tablet report, sending...
I20260806 04:37:09.877851 1715 ts_manager.cc:194] Registered new tserver with Master: 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805)
I20260806 04:37:09.878650 1715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:35449
W20260806 04:37:09.982048 2228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:09.982288 2228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:09.982327 2228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:09.986366 2228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:09.986420 2228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:09.986495 2228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:37:09.991775 2228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:39219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=46215
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:09.992859 2228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:09.993894 2228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:10.003340 2237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.003374 2236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.003386 2240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:10.003569 2228 server_base.cc:1034] running on GCE node
I20260806 04:37:10.004016 2228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:10.004735 2228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:10.005895 2228 hybrid_clock.cc:648] HybridClock initialized: now 1785991030005869 us; error 43 us; skew 500 ppm
Aug 06 04:37:10 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:10.008638 2228 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:37:10.009541 2228 webserver.cc:460] Webserver started at http://127.12.15.195:46215/ using document root <none> and password file <none>
I20260806 04:37:10.010110 2228 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:10.010190 2228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:10.014344 2228 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:37:10.016606 2252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.017930 2228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:37:10.018050 2228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5fe3f77e533640989a3637c0162e3290"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "418f5297023bb94a9673f81ce385e198"
server_key_iv: "a61f848bf35ef8f8c000f45d57791511"
server_key_version: "encryptionkey@0"
I20260806 04:37:10.018612 2228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:10.035624 2228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:10.038861 2228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:10.039047 2228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:10.039700 2228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:10.040598 2228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:10.040652 2228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.040707 2228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:10.040733 2228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.049542 2228 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39219
I20260806 04:37:10.049572 2430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39219 every 8 connection(s)
I20260806 04:37:10.050534 2228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:37:10.054674 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 2228
Aug 06 04:37:10 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:10.065142 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.055356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.12.15.254:38533)
0806 04:37:10.055630 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.055635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.055710 (+ 75us) server_negotiation.cc:407] Connection header received
0806 04:37:10.055838 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.055841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.055883 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.055956 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:10.056908 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.057656 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.058920 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.059250 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.059351 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.061840 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:10.061843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:10.061845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:10.061880 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:10.063365 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:10.063825 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:10.063828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:10.063829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:10.063931 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:10.064211 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:10.064214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:10.064216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:10.064458 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:10.064616 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.064834 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.064959 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
I20260806 04:37:10.065850 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.052407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:35421 (local address 127.12.15.254:38533)
0806 04:37:10.052586 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.052591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.053648 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 04:37:10.055054 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.055057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.055108 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.055192 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:10.056458 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.057103 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.058256 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.058472 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.058569 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.061636 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:10.061640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:10.061643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:10.061672 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:10.063218 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:10.063797 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:10.063801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:10.063803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:10.063907 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:10.064317 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:10.064320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:10.064321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:10.064559 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:10.064662 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.065576 (+ 914us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.065689 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 04:37:10.066567 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.052906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:35421)
0806 04:37:10.053498 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.053543 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:10.054842 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:10.055406 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:10.055415 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:10.055807 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:10.056296 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.056310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.057227 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.057231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.058129 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.058140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.058613 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.058617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.058800 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.059399 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:10.059420 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:10.061460 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:10.063338 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:10.063344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:10.063361 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:10.063658 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:10.064010 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:10.064014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:10.064015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:10.064176 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:10.064673 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:10.064680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:10.065005 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 04:37:10.065223 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 04:37:10.065537 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":134,"threads_started":1}
I20260806 04:37:10.067624 2433 heartbeater.cc:347] Connected to a master server at 127.12.15.254:38533
I20260806 04:37:10.067865 2433 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:10.068403 2433 heartbeater.cc:511] Master 127.12.15.254:38533 requested a full tablet report, sending...
I20260806 04:37:10.069414 1715 ts_manager.cc:194] Registered new tserver with Master: 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
I20260806 04:37:10.069967 1715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:35421
I20260806 04:37:10.079212 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:37:10.083715 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.079853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.12.15.254:38533)
0806 04:37:10.079996 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.080000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.080124 (+ 124us) server_negotiation.cc:407] Connection header received
0806 04:37:10.080264 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.080266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.080434 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.080522 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:10.080766 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.081287 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.082137 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.082436 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.082516 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.082859 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:10.083268 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:37:10.083364 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.083495 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.083545 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:37:10.084425 1715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:37:10.087878 1715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:37:10.096499 1903 tablet_service.cc:1467] Processing CreateTablet for tablet 09bfa63c29f14e678b17e64566a47ca8 (DEFAULT_TABLE table=test-table [id=ba6675cb9d894f6db01b490e6a4bf2b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:10.097239 1903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09bfa63c29f14e678b17e64566a47ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:10.100214 1839 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Bootstrap starting.
I20260806 04:37:10.100251 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.095296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.12.15.194:34805)
0806 04:37:10.095664 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.095683 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.095708 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:37:10.095793 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.095799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.095987 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.096155 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:10.096392 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.097619 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.098969 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.099507 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.099621 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.099882 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.100012 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.100108 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":137,"threads_started":1}
I20260806 04:37:10.101140 1839 tablet_bootstrap.cc:654] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:10.101200 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.095208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39219 (local address 127.0.0.1:48120)
0806 04:37:10.095502 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.095515 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:10.095604 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:10.096295 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:10.096297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:10.096313 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:10.096413 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.096420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.097600 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.097603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.098608 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.098620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.099675 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.099679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.099772 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.100932 (+ 1160us) client_negotiation.cc:769] Sending connection context
0806 04:37:10.100993 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 04:37:10.101080 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":1122,"thread_start_us":110,"threads_started":1}
I20260806 04:37:10.100234 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.094944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34805 (local address 127.0.0.1:56528)
0806 04:37:10.095086 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.095101 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:10.095179 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:10.096144 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:10.096148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:10.096166 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:10.096280 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.096286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.097782 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.097785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.098862 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.098869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.099630 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.099633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.099840 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.099889 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 04:37:10.099978 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 04:37:10.100063 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":59}
I20260806 04:37:10.101894 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.095469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.12.15.195:39219)
0806 04:37:10.095854 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.095865 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.095887 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:37:10.095956 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.095962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.096149 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.096529 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:10.096674 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.097470 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.098739 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.099522 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.099842 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.099982 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.101653 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.101759 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":164,"threads_started":1}
I20260806 04:37:10.103006 1839 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: No bootstrap required, opened a new log
I20260806 04:37:10.103106 1839 ts_tablet_manager.cc:1397] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:37:10.103557 1839 raft_consensus.cc:348] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.103677 1839 raft_consensus.cc:374] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:10.103715 1839 raft_consensus.cc:729] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2029162826452f8b73701183c04480, State: Initialized, Role: FOLLOWER
I20260806 04:37:10.103844 1839 consensus_queue.cc:260] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.104099 1839 ts_tablet_manager.cc:1428] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 04:37:10.104278 2115 tablet_service.cc:1467] Processing CreateTablet for tablet 09bfa63c29f14e678b17e64566a47ca8 (DEFAULT_TABLE table=test-table [id=ba6675cb9d894f6db01b490e6a4bf2b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:10.104624 2326 tablet_service.cc:1467] Processing CreateTablet for tablet 09bfa63c29f14e678b17e64566a47ca8 (DEFAULT_TABLE table=test-table [id=ba6675cb9d894f6db01b490e6a4bf2b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:10.105345 2115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09bfa63c29f14e678b17e64566a47ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:10.105522 2326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09bfa63c29f14e678b17e64566a47ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:10.112167 2459 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Bootstrap starting.
I20260806 04:37:10.112167 2458 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Bootstrap starting.
I20260806 04:37:10.114576 2459 tablet_bootstrap.cc:654] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:10.114990 2458 tablet_bootstrap.cc:654] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:10.115664 2458 log.cc:824] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:10.115665 2459 log.cc:824] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:10.117653 2459 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: No bootstrap required, opened a new log
I20260806 04:37:10.117866 2459 ts_tablet_manager.cc:1397] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:37:10.117885 2458 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: No bootstrap required, opened a new log
I20260806 04:37:10.118074 2458 ts_tablet_manager.cc:1397] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:37:10.120996 2459 raft_consensus.cc:348] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.121038 2458 raft_consensus.cc:348] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.121239 2459 raft_consensus.cc:374] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:10.121251 2458 raft_consensus.cc:374] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:10.121300 2459 raft_consensus.cc:729] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe3f77e533640989a3637c0162e3290, State: Initialized, Role: FOLLOWER
I20260806 04:37:10.121310 2458 raft_consensus.cc:729] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114816b98ffc490b92e6defd6a479584, State: Initialized, Role: FOLLOWER
I20260806 04:37:10.121734 2459 consensus_queue.cc:260] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.121739 2458 consensus_queue.cc:260] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.122442 2433 heartbeater.cc:503] Master 127.12.15.254:38533 was elected leader, sending a full tablet report...
I20260806 04:37:10.122740 2459 ts_tablet_manager.cc:1428] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260806 04:37:10.122802 2458 ts_tablet_manager.cc:1428] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260806 04:37:10.123623 2219 heartbeater.cc:503] Master 127.12.15.254:38533 was elected leader, sending a full tablet report...
I20260806 04:37:10.151840 2463 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:10.152055 2463 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
W20260806 04:37:10.152969 2001 tablet.cc:2369] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:37:10.153225 2463 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), da2029162826452f8b73701183c04480 (127.12.15.193:37881)
I20260806 04:37:10.156596 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.153413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34805 (local address 127.12.15.195:34225)
0806 04:37:10.153557 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.153573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:10.153677 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:10.153957 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:10.153960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:10.153979 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:10.154093 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.154099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.154803 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.154806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.155558 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.155568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.156179 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.156182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.156289 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.156310 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 04:37:10.156394 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:37:10.156463 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 04:37:10.156599 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.153546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:34225 (local address 127.12.15.194:34805)
0806 04:37:10.153698 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.153702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.153721 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:37:10.153776 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.153780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.153834 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.153911 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:10.154204 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.154699 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.155682 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.156071 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.156176 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.156233 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.156422 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.156490 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 04:37:10.157068 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.153799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:44973 (local address 127.12.15.193:37881)
0806 04:37:10.153917 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.153921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.154085 (+ 164us) server_negotiation.cc:407] Connection header received
0806 04:37:10.154181 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.154184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.154240 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.154317 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:10.154575 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.155092 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.156089 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.156560 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.156670 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.156822 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.156895 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.156964 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 04:37:10.157075 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.153650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:37881 (local address 127.12.15.195:44973)
0806 04:37:10.153996 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.154008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:10.154095 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:10.154339 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:10.154341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:10.154357 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:10.154465 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.154470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.155196 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.155199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.155959 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:10.155967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.156681 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.156685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:10.156774 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.156794 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:37:10.156871 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 04:37:10.156936 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":145,"threads_started":1}
I20260806 04:37:10.157315 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09bfa63c29f14e678b17e64566a47ca8" candidate_uuid: "5fe3f77e533640989a3637c0162e3290" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114816b98ffc490b92e6defd6a479584" is_pre_election: true
I20260806 04:37:10.157641 2150 raft_consensus.cc:2454] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3f77e533640989a3637c0162e3290 in term 0.
I20260806 04:37:10.157815 1936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09bfa63c29f14e678b17e64566a47ca8" candidate_uuid: "5fe3f77e533640989a3637c0162e3290" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da2029162826452f8b73701183c04480" is_pre_election: true
I20260806 04:37:10.158087 1936 raft_consensus.cc:2454] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fe3f77e533640989a3637c0162e3290 in term 0.
I20260806 04:37:10.158109 2253 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290; no voters:
I20260806 04:37:10.158366 2463 raft_consensus.cc:2781] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:37:10.158461 2463 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:37:10.158495 2463 raft_consensus.cc:3037] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:10.159533 2463 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.159883 2463 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [CANDIDATE]: Term 1 election: Requested vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), da2029162826452f8b73701183c04480 (127.12.15.193:37881)
I20260806 04:37:10.160244 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09bfa63c29f14e678b17e64566a47ca8" candidate_uuid: "5fe3f77e533640989a3637c0162e3290" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114816b98ffc490b92e6defd6a479584"
I20260806 04:37:10.160303 1936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09bfa63c29f14e678b17e64566a47ca8" candidate_uuid: "5fe3f77e533640989a3637c0162e3290" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da2029162826452f8b73701183c04480"
I20260806 04:37:10.160400 2150 raft_consensus.cc:3037] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:10.160449 1936 raft_consensus.cc:3037] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:10.161669 2150 raft_consensus.cc:2454] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3f77e533640989a3637c0162e3290 in term 1.
I20260806 04:37:10.161669 1936 raft_consensus.cc:2454] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fe3f77e533640989a3637c0162e3290 in term 1.
I20260806 04:37:10.161976 2253 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290; no voters:
I20260806 04:37:10.162263 2463 raft_consensus.cc:2781] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:10.162500 2463 raft_consensus.cc:686] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe3f77e533640989a3637c0162e3290, State: Running, Role: LEADER
I20260806 04:37:10.162832 2463 consensus_queue.cc:237] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.166184 1714 catalog_manager.cc:5495] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fe3f77e533640989a3637c0162e3290 (127.12.15.195). New cstate: current_term: 1 leader_uuid: "5fe3f77e533640989a3637c0162e3290" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } health_report { overall_health: UNKNOWN } } }
I20260806 04:37:10.186273 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.182907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.12.15.195:39219)
0806 04:37:10.183034 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:10.183039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:10.183053 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:37:10.183165 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:10.183167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:10.183224 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:10.183316 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:10.183541 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.184040 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.184861 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:10.185064 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:10.185153 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:10.185463 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:10.185893 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:37:10.186000 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:10.186082 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 04:37:10.186137 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:37:10.192394 1936 raft_consensus.cc:1261] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Refusing update from remote peer 5fe3f77e533640989a3637c0162e3290: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:10.192399 2150 raft_consensus.cc:1261] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Refusing update from remote peer 5fe3f77e533640989a3637c0162e3290: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:10.193110 2468 consensus_queue.cc:1035] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [LEADER]: Connected to new peer: Peer: permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:10.193282 2463 consensus_queue.cc:1035] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:10.199621 2475 mvcc.cc:205] Tried to move back new op lower bound from 7315419259660595200 to 7315419259549548544. Current Snapshot: MvccSnapshot[applied={T|T < 7315419259660595200}]
I20260806 04:37:10.202415 2476 mvcc.cc:205] Tried to move back new op lower bound from 7315419259660595200 to 7315419259549548544. Current Snapshot: MvccSnapshot[applied={T|T < 7315419259660595200}]
I20260806 04:37:10.208014 1715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:10.208159 1715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:10.210469 1715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Sending DeleteTablet for 3 replicas of tablet 09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:10.211158 1903 tablet_service.cc:1514] Processing DeleteTablet for tablet 09bfa63c29f14e678b17e64566a47ca8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:10 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:60836
I20260806 04:37:10.211179 2326 tablet_service.cc:1514] Processing DeleteTablet for tablet 09bfa63c29f14e678b17e64566a47ca8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:10 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:48120
I20260806 04:37:10.211211 2115 tablet_service.cc:1514] Processing DeleteTablet for tablet 09bfa63c29f14e678b17e64566a47ca8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:10 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:56528
I20260806 04:37:10.211516 2487 tablet_replica.cc:330] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: stopping tablet replica
I20260806 04:37:10.211658 2487 raft_consensus.cc:2229] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:10.211791 2487 raft_consensus.cc:2258] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:10.211884 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 1771
I20260806 04:37:10.225096 2490 tablet_replica.cc:330] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: stopping tablet replica
I20260806 04:37:10.225540 2490 raft_consensus.cc:2229] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:37:10.225963 2490 raft_consensus.cc:2258] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:10.226366 2491 tablet_replica.cc:330] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: stopping tablet replica
I20260806 04:37:10.226675 2491 raft_consensus.cc:2229] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:10.226997 2491 raft_consensus.cc:2258] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:10.227825 2490 ts_tablet_manager.cc:1905] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:10.228488 2491 ts_tablet_manager.cc:1905] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:10.230726 2490 ts_tablet_manager.cc:1918] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:10.230842 2490 log.cc:1196] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:10.231119 2491 ts_tablet_manager.cc:1918] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:10.231158 2490 ts_tablet_manager.cc:1939] T 09bfa63c29f14e678b17e64566a47ca8 P 5fe3f77e533640989a3637c0162e3290: Deleting consensus metadata
I20260806 04:37:10.231227 2491 log.cc:1196] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:10.231566 2491 ts_tablet_manager.cc:1939] T 09bfa63c29f14e678b17e64566a47ca8 P 114816b98ffc490b92e6defd6a479584: Deleting consensus metadata
I20260806 04:37:10.232523 1688 catalog_manager.cc:4841] TS 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): tablet 09bfa63c29f14e678b17e64566a47ca8 (table test-table [id=ba6675cb9d894f6db01b490e6a4bf2b4]) successfully deleted
I20260806 04:37:10.232771 1685 catalog_manager.cc:4841] TS 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): tablet 09bfa63c29f14e678b17e64566a47ca8 (table test-table [id=ba6675cb9d894f6db01b490e6a4bf2b4]) successfully deleted
W20260806 04:37:10.233338 1685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:37881 (error 108)
I20260806 04:37:10.233881 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 2007
I20260806 04:37:10.237816 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.234082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:37881 (local address 127.0.0.1:60850)
0806 04:37:10.237638 (+ 3556us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.237723 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3449}
W20260806 04:37:10.242652 1685 catalog_manager.cc:4584] TS da2029162826452f8b73701183c04480 (127.12.15.193:37881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09bfa63c29f14e678b17e64566a47ca8: Network error: Client connection negotiation failed: client connection to 127.12.15.193:37881: connect: Connection refused (error 111)
I20260806 04:37:10.249133 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 2228
I20260806 04:37:10.259765 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 1661
I20260806 04:37:10.267906 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38533
--webserver_interface=127.12.15.254
--webserver_port=33753
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:10.381230 2495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:10.381510 2495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:10.381573 2495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:10.385874 2495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:10.385936 2495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:10.385965 2495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:10.385988 2495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:10.386016 2495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:10.391264 2495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:38533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=33753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:10.392287 2495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:10.393301 2495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:10.401496 2503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.401486 2500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:10.401532 2495 server_base.cc:1034] running on GCE node
W20260806 04:37:10.401486 2501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:10.402248 2495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:10.403244 2495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:10.404399 2495 hybrid_clock.cc:648] HybridClock initialized: now 1785991030404385 us; error 28 us; skew 500 ppm
Aug 06 04:37:10 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:10.407012 2495 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:10.407856 2495 webserver.cc:460] Webserver started at http://127.12.15.254:33753/ using document root <none> and password file <none>
I20260806 04:37:10.408411 2495 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:10.408478 2495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:10.412133 2495 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:37:10.414271 2512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.415496 2495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:10.415607 2495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "50f858fa3f954036b5b61a5c9ef552c4"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "852119160e3485eb4b20ae6caca648a5"
server_key_iv: "4106f9cee35b2c2fdb0e16079edbd67e"
server_key_version: "encryptionkey@0"
I20260806 04:37:10.416141 2495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:10.427632 2495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:10.430370 2495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:10.430537 2495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:10.437757 2495 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:38533
I20260806 04:37:10.438027 2610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:38533 every 8 connection(s)
I20260806 04:37:10.438957 2495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:10.443408 2611 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:37:10.444661 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 2495
I20260806 04:37:10.445067 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:37881
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:38533
--webserver_port=38883
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:10.445145 2611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap starting.
I20260806 04:37:10.462462 2611 log.cc:824] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:10.473852 2611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:37:10.474197 2611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4: Bootstrap complete.
I20260806 04:37:10.478163 2611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:10.478559 2611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f858fa3f954036b5b61a5c9ef552c4, State: Initialized, Role: FOLLOWER
I20260806 04:37:10.479031 2611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:10.479156 2611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:37:10.479223 2611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:37:10.479318 2611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 2 FOLLOWER]: Advancing to term 3
I20260806 04:37:10.481195 2611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:10.481508 2611 leader_election.cc:302] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50f858fa3f954036b5b61a5c9ef552c4; no voters:
I20260806 04:37:10.481819 2611 leader_election.cc:288] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 04:37:10.481954 2615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 04:37:10.482208 2615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [term 3 LEADER]: Becoming Leader. State: Replica: 50f858fa3f954036b5b61a5c9ef552c4, State: Running, Role: LEADER
I20260806 04:37:10.482540 2615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } }
I20260806 04:37:10.482965 2611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:10.483541 2615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f858fa3f954036b5b61a5c9ef552c4. Latest consensus state: current_term: 3 leader_uuid: "50f858fa3f954036b5b61a5c9ef552c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } } }
I20260806 04:37:10.483660 2615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:10.484354 2618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "50f858fa3f954036b5b61a5c9ef552c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50f858fa3f954036b5b61a5c9ef552c4" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38533 } } }
I20260806 04:37:10.484459 2618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f858fa3f954036b5b61a5c9ef552c4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:10.485298 2625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:10.491211 2625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:10.491891 2625 catalog_manager.cc:1259] Loaded cluster ID: 2380d2fb99c34d4daa0256e7d3b346d2
I20260806 04:37:10.491954 2625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 04:37:10.493112 2625 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb4830b99ad google::LogMessage::Fail() at ??:0
@ 0x7fb4830bc88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb4830b94de google::LogMessage::Flush() at ??:0
@ 0x7fb4830baa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb48d9f8fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fb48d9f9d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb48d9f6579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fb48da3a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb48530b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb48530bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fb48530d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb4852fc512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb4869406db start_thread at ??:0
@ 0x7fb481bb761f clone at ??:0
W20260806 04:37:10.597656 2613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:10.598057 2613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:10.598151 2613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:10.604274 2613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:10.604391 2613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:10.604550 2613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:37:10.612641 2613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:37881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=38883
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:10.614260 2613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:10.615670 2613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:10.627249 2677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.627202 2673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.627208 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:10.627439 2613 server_base.cc:1034] running on GCE node
I20260806 04:37:10.628131 2613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:10.628822 2613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:10.629999 2613 hybrid_clock.cc:648] HybridClock initialized: now 1785991030629966 us; error 51 us; skew 500 ppm
Aug 06 04:37:10 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:10.632748 2613 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:37:10.633674 2613 webserver.cc:460] Webserver started at http://127.12.15.193:38883/ using document root <none> and password file <none>
I20260806 04:37:10.634232 2613 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:10.634294 2613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:10.638449 2613 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:37:10.640870 2686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.642284 2613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:37:10.642422 2613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "da2029162826452f8b73701183c04480"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "bb603a478a4b6031d1692217410a426d"
server_key_iv: "4fcd3f86630e31504532eed8192692cd"
server_key_version: "encryptionkey@0"
I20260806 04:37:10.643003 2613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:10.663097 2613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:10.666651 2613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:10.666846 2613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:10.667496 2613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:10.668751 2699 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 04:37:10.674593 2613 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 04:37:10.674672 2613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 04:37:10.674758 2613 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 04:37:10.677654 2613 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 04:37:10.677718 2613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260806 04:37:10.678187 2699 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Bootstrap starting.
I20260806 04:37:10.688948 2613 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:37881
I20260806 04:37:10.689947 2613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:37:10.690737 2864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:37881 every 8 connection(s)
I20260806 04:37:10.693773 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 2613
I20260806 04:37:10.694262 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:34805
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:38533
--webserver_port=39803
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:10.702039 2699 log.cc:824] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:10.711686 2699 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:37:10.712116 2699 tablet_bootstrap.cc:492] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Bootstrap complete.
I20260806 04:37:10.713508 2699 ts_tablet_manager.cc:1397] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Time spent bootstrapping tablet: real 0.036s user 0.027s sys 0.005s
I20260806 04:37:10.716902 2699 raft_consensus.cc:348] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.717468 2699 raft_consensus.cc:729] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2029162826452f8b73701183c04480, State: Initialized, Role: FOLLOWER
I20260806 04:37:10.718022 2699 consensus_queue.cc:260] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:10.719090 2699 ts_tablet_manager.cc:1428] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 04:37:10.739908 2867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:37:10.692413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:40451)
0806 04:37:10.693064 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.693127 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:10.694305 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:10.739029 (+ 44724us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":163,"threads_started":1}
W20260806 04:37:10.740533 2865 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:38533 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.254:38533: Client connection negotiation failed: client connection to 127.12.15.254:38533: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 04:37:10.741257 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.741010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:49227)
0806 04:37:10.741134 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.741200 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 04:37:10.742022 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.741748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:45053)
0806 04:37:10.741885 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.741956 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 04:37:10.742314 2865 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:37:10.821120 2868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:10.821374 2868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:10.821434 2868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:10.825448 2868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:10.825502 2868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:10.825580 2868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:37:10.830727 2868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:34805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=39803
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:10.831722 2868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:10.832763 2868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:10.842160 2884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.842216 2886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:10.842228 2888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:10.842335 2868 server_base.cc:1034] running on GCE node
I20260806 04:37:10.842804 2868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:10.843425 2868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:10.844584 2868 hybrid_clock.cc:648] HybridClock initialized: now 1785991030844557 us; error 42 us; skew 500 ppm
Aug 06 04:37:10 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:10.847203 2868 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:37:10.848064 2868 webserver.cc:460] Webserver started at http://127.12.15.194:39803/ using document root <none> and password file <none>
I20260806 04:37:10.848636 2868 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:10.848707 2868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:10.852635 2868 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 04:37:10.854547 2901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.855686 2868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:10.855793 2868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "114816b98ffc490b92e6defd6a479584"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "aae052d0502da18137e60b62ce0b9b90"
server_key_iv: "32b8f45adef47cfbe4816d21fc879364"
server_key_version: "encryptionkey@0"
I20260806 04:37:10.856333 2868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:10.876971 2868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:10.880400 2868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:10.880610 2868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:10.881229 2868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:10.882102 2868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:10.882151 2868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.882195 2868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:10.882223 2868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:10.890694 2868 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:34805
I20260806 04:37:10.890739 3081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:34805 every 8 connection(s)
I20260806 04:37:10.891698 2868 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:37:10.893850 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 2868
I20260806 04:37:10.894328 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:39219
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:38533
--webserver_port=46215
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:10.895917 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.893920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:46701)
0806 04:37:10.894489 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.894631 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":157,"threads_started":1}
W20260806 04:37:10.896389 3084 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:38533 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.254:38533: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
I20260806 04:37:10.896998 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.896787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:36347)
0806 04:37:10.896893 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.896942 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 04:37:10.897588 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:10.897412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:39345)
0806 04:37:10.897495 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:10.897534 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 04:37:10.897737 3084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:37:11.012253 3090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:11.012586 3090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:11.012634 3090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:11.016672 3090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:11.016741 3090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:11.016804 3090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:37:11.022161 3090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:39219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=46215
--tserver_master_addrs=127.12.15.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:11.023254 3090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:11.024314 3090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:11.033947 3095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:11.033993 3097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:11.033991 3103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:11.034092 3090 server_base.cc:1034] running on GCE node
I20260806 04:37:11.034559 3090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:11.035171 3090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:11.036315 3090 hybrid_clock.cc:648] HybridClock initialized: now 1785991031036287 us; error 42 us; skew 500 ppm
Aug 06 04:37:11 dist-test-slave-xt4k krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991031, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:11.039073 3090 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:37:11.040004 3090 webserver.cc:460] Webserver started at http://127.12.15.195:46215/ using document root <none> and password file <none>
I20260806 04:37:11.040596 3090 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:11.040660 3090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:11.044497 3090 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 04:37:11.046550 3113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:11.047825 3090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:11.047945 3090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5fe3f77e533640989a3637c0162e3290"
format_stamp: "Formatted at 2026-08-06 04:37:08 on dist-test-slave-xt4k"
server_key: "418f5297023bb94a9673f81ce385e198"
server_key_iv: "a61f848bf35ef8f8c000f45d57791511"
server_key_version: "encryptionkey@0"
I20260806 04:37:11.048511 3090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:11.070495 3090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:11.073457 3090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:11.073649 3090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:11.074244 3090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:11.075083 3090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:11.075131 3090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:11.075187 3090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:11.075214 3090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:11.083952 3090 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39219
I20260806 04:37:11.083964 3292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39219 every 8 connection(s)
I20260806 04:37:11.084988 3090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:37:11.088799 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:11.087094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:40393)
0806 04:37:11.087552 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:11.087664 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
W20260806 04:37:11.089193 3293 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:38533 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.254:38533: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
I20260806 04:37:11.089749 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:11.089555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:37207)
0806 04:37:11.089652 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:11.089697 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 04:37:11.090250 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:11.090087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:60517)
0806 04:37:11.090166 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:11.090204 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 04:37:11.090382 3293 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 04:37:11.092303 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 3090
I20260806 04:37:11.743361 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:11.742901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:49165)
0806 04:37:11.743176 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:11.743237 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
I20260806 04:37:11.898669 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:11.898263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:36259)
0806 04:37:11.898510 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:11.898563 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
I20260806 04:37:12.091352 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.090922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:59369)
0806 04:37:12.091191 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:12.091251 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260806 04:37:12.303404 3306 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:12.303689 3306 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:12.305016 3306 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
Aug 06 04:37:12 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.194@KRBTEST.COM
Aug 06 04:37:12 dist-test-slave-xt4k krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991030, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.195@KRBTEST.COM
I20260806 04:37:12.318550 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.305457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:34049 (local address 127.12.15.195:39219)
0806 04:37:12.306053 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:12.306083 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:12.306117 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:37:12.306736 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:12.306741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:12.307037 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:12.307293 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:12.308556 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.309508 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.310678 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.311134 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.311280 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:12.314317 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:12.314321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:12.314333 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:12.314360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:12.315942 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:12.316514 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:12.316521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:12.316527 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:12.316650 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:12.317026 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:12.317029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:12.317030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:12.317429 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:12.317548 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:12.318112 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 04:37:12.318373 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":101,"threads_started":1}
I20260806 04:37:12.318533 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.305255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34805 (local address 127.12.15.193:51709)
0806 04:37:12.305856 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:12.305869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:12.305972 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:12.307620 (+ 1648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:12.307623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:12.307862 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:12.308383 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:12.308397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.309635 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.309638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:12.310560 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:12.310570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.311242 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.311245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:12.311458 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:12.312097 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:12.312122 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:12.313786 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:12.315999 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:12.316006 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:12.316021 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:12.316393 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:12.316743 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:12.316747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:12.316749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:12.316917 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:12.317546 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:12.317552 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:12.317791 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 04:37:12.317956 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 04:37:12.318319 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":489,"spinlock_wait_cycles":24960,"thread_start_us":82,"threads_started":1}
I20260806 04:37:12.318557 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.305400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:51709 (local address 127.12.15.194:34805)
0806 04:37:12.305828 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:12.305850 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:12.305955 (+ 105us) server_negotiation.cc:407] Connection header received
0806 04:37:12.306677 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:12.306683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:12.307013 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:12.307260 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:12.308540 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.309501 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.310674 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.311129 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.311277 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:12.314015 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:12.314021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:12.314039 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:12.314074 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:12.315868 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:12.316526 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:12.316531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:12.316535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:12.316650 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:12.317017 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:12.317020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:12.317021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:12.317429 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:12.317547 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:12.318121 (+ 574us) server_negotiation.cc:299] Negotiation successful
0806 04:37:12.318365 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":73,"threads_started":1}
I20260806 04:37:12.318555 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.305256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39219 (local address 127.12.15.193:34049)
0806 04:37:12.305558 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:12.305574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:12.305683 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:12.307376 (+ 1693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:12.307382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:12.307874 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:12.308380 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:12.308404 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.309642 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.309644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:12.310563 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:12.310574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:12.311257 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:12.311261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:12.311460 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:12.312097 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:12.312126 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:12.314141 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:12.316057 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:12.316060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:12.316061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:12.316384 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:12.316743 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:12.316747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:12.316749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:12.316924 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:12.317546 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:12.317553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:12.317792 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 04:37:12.317957 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 04:37:12.318395 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":404,"spinlock_wait_cycles":52480,"thread_start_us":84,"threads_started":1}
W20260806 04:37:12.321336 2691 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
W20260806 04:37:12.321516 2688 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:12.321588 2688 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da2029162826452f8b73701183c04480; no voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290
I20260806 04:37:12.321822 3306 raft_consensus.cc:2726] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:37:12.744661 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.744332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:41867)
0806 04:37:12.744510 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:12.744574 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 04:37:12.899829 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:12.899424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:37745)
0806 04:37:12.899666 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:12.899727 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":74,"threads_started":1}
I20260806 04:37:13.092619 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:13.092157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:52585)
0806 04:37:13.092458 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:13.092519 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260806 04:37:13.745932 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:13.745498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:60189)
0806 04:37:13.745771 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:13.745832 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260806 04:37:13.901086 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:13.900627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:52195)
0806 04:37:13.900928 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:13.900985 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
I20260806 04:37:14.093781 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:14.093342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:34481)
0806 04:37:14.093613 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:14.093672 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
I20260806 04:37:14.208619 3317 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:14.208747 3317 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:14.209120 3317 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
W20260806 04:37:14.209972 2691 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
W20260806 04:37:14.210103 2688 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:14.210162 2688 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da2029162826452f8b73701183c04480; no voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290
I20260806 04:37:14.210290 3317 raft_consensus.cc:2726] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:37:14.747140 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:14.746691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:50281)
0806 04:37:14.746983 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:14.747040 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":74,"threads_started":1}
I20260806 04:37:14.902254 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:14.901823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:46925)
0806 04:37:14.902098 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:14.902157 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
I20260806 04:37:15.094894 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:15.094468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:38913)
0806 04:37:15.094741 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:15.094799 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260806 04:37:15.748446 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:15.747985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:52297)
0806 04:37:15.748253 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:15.748309 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":72,"threads_started":1}
I20260806 04:37:15.903463 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:15.903038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:59403)
0806 04:37:15.903309 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:15.903366 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
I20260806 04:37:16.069882 3324 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:16.070225 3324 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:16.070680 3324 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
W20260806 04:37:16.071480 2691 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
W20260806 04:37:16.071592 2688 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:16.071677 2688 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da2029162826452f8b73701183c04480; no voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290
I20260806 04:37:16.071817 3324 raft_consensus.cc:2726] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:37:16.096143 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:16.095670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:41771)
0806 04:37:16.095974 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:16.096050 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":106,"threads_started":1}
I20260806 04:37:16.749706 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:16.749250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:46309)
0806 04:37:16.749532 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:16.749603 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260806 04:37:16.904796 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:16.904314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:54745)
0806 04:37:16.904632 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:16.904696 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":79,"threads_started":1}
I20260806 04:37:17.097496 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:17.097031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:54551)
0806 04:37:17.097331 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:17.097397 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260806 04:37:17.751041 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:17.750569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:50065)
0806 04:37:17.750862 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:17.750928 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":81,"threads_started":1}
I20260806 04:37:17.906224 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:17.905702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:39509)
0806 04:37:17.906055 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:17.906120 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":93,"threads_started":1}
I20260806 04:37:18.098924 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:18.098387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:39401)
0806 04:37:18.098762 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:18.098827 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":96,"threads_started":1}
I20260806 04:37:18.457732 3344 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:18.457890 3344 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:18.458410 3344 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
W20260806 04:37:18.459213 2691 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
W20260806 04:37:18.459316 2688 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:18.459408 2688 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da2029162826452f8b73701183c04480; no voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290
I20260806 04:37:18.459568 3344 raft_consensus.cc:2726] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:37:18.752511 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:18.752018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:56125)
0806 04:37:18.752319 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:18.752408 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
I20260806 04:37:18.907573 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:18.907124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:38007)
0806 04:37:18.907408 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:18.907474 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260806 04:37:19.100221 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:19.099792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:47079)
0806 04:37:19.100061 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:19.100124 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260806 04:37:19.753666 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:19.753258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:52775)
0806 04:37:19.753506 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:19.753564 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260806 04:37:19.908793 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:19.908327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:38005)
0806 04:37:19.908628 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:19.908688 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":76,"threads_started":1}
I20260806 04:37:20.101634 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:20.101097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:55253)
0806 04:37:20.101454 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:20.101524 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260806 04:37:20.754895 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:20.754460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:37677)
0806 04:37:20.754732 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:20.754792 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260806 04:37:20.910162 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:20.909668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:43417)
0806 04:37:20.909990 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:20.910060 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":82,"threads_started":1}
I20260806 04:37:21.102880 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:21.102442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:42973)
0806 04:37:21.102717 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:21.102781 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
I20260806 04:37:21.756160 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:21.755744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:44095)
0806 04:37:21.756001 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:21.756060 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260806 04:37:21.911340 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:21.910924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:60291)
0806 04:37:21.911185 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:21.911241 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260806 04:37:22.104177 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:22.103696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:52595)
0806 04:37:22.104010 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:22.104072 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
I20260806 04:37:22.757477 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:22.756971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:55145)
0806 04:37:22.757287 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:22.757349 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":110,"threads_started":1}
I20260806 04:37:22.789726 3372 raft_consensus.cc:482] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:22.789878 3372 raft_consensus.cc:504] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "114816b98ffc490b92e6defd6a479584" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34805 } } peers { permanent_uuid: "5fe3f77e533640989a3637c0162e3290" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39219 } } peers { permanent_uuid: "da2029162826452f8b73701183c04480" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 37881 } }
I20260806 04:37:22.790326 3372 leader_election.cc:288] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805), 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219)
W20260806 04:37:22.791213 2691 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fe3f77e533640989a3637c0162e3290 (127.12.15.195:39219): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
W20260806 04:37:22.791328 2688 leader_election.cc:341] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 114816b98ffc490b92e6defd6a479584 (127.12.15.194:34805): Not found: Tablet not found: 09bfa63c29f14e678b17e64566a47ca8
I20260806 04:37:22.791411 2688 leader_election.cc:302] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da2029162826452f8b73701183c04480; no voters: 114816b98ffc490b92e6defd6a479584, 5fe3f77e533640989a3637c0162e3290
I20260806 04:37:22.791571 3372 raft_consensus.cc:2726] T 09bfa63c29f14e678b17e64566a47ca8 P da2029162826452f8b73701183c04480 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:37:22.912628 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:22.912159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:36377)
0806 04:37:22.912442 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:22.912504 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260806 04:37:23.105473 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:23.105020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:52347)
0806 04:37:23.105302 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:23.105366 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260806 04:37:23.758718 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:23.758259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:44979)
0806 04:37:23.758549 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:23.758615 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260806 04:37:23.913977 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:23.913496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:36755)
0806 04:37:23.913810 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:23.913874 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":86,"threads_started":1}
I20260806 04:37:24.106781 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:24.106296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:33345)
0806 04:37:24.106609 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:24.106673 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260806 04:37:24.759966 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:24.759503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:49065)
0806 04:37:24.759807 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:24.759866 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":103,"threads_started":1}
I20260806 04:37:24.915267 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:24.914825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:53709)
0806 04:37:24.915094 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:24.915164 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":79,"threads_started":1}
I20260806 04:37:25.108088 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:25.107616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:42761)
0806 04:37:25.107914 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:25.107984 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
I20260806 04:37:25.761389 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:25.760989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.193:40493)
0806 04:37:25.761236 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:25.761291 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":90,"threads_started":1}
I20260806 04:37:25.916667 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:25.916114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.194:49191)
0806 04:37:25.916435 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:25.916543 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":72,"threads_started":1}
I20260806 04:37:26.100926 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 2613
I20260806 04:37:26.109479 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:26.109082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:38533 (local address 127.12.15.195:49527)
0806 04:37:26.109329 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:26.109386 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
I20260806 04:37:26.111975 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 2868
I20260806 04:37:26.125264 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 3090
2026-08-06T04:37:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19305 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:37:26 dist-test-slave-xt4k krb5kdc[3397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:37:26 dist-test-slave-xt4k krb5kdc[3397](info): set up 2 sockets
Aug 06 04:37:26 dist-test-slave-xt4k krb5kdc[3397](info): commencing operation
krb5kdc: starting...
W20260806 04:37:27.277370 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:37:27 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:37:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:37:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:37:27.402690 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43745
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:43745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:27.513505 3415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:27.513772 3415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:27.513827 3415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:27.517753 3415 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 04:37:27.517796 3415 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 04:37:27.517817 3415 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 04:37:27.517833 3415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:27.517858 3415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:27.517890 3415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:27.517917 3415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:27.517946 3415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:27.523062 3415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:43745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:27.524048 3415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:27.525059 3415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:37:27.533447 3415 server_base.cc:1034] running on GCE node
W20260806 04:37:27.533447 3423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:27.533452 3425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:27.533612 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:27.534286 3415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:27.535372 3415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:27.536537 3415 hybrid_clock.cc:648] HybridClock initialized: now 1785991047536514 us; error 32 us; skew 500 ppm
Aug 06 04:37:27 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991047, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:27.539239 3415 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:27.540114 3415 webserver.cc:460] Webserver started at http://127.12.15.254:37593/ using document root <none> and password file <none>
I20260806 04:37:27.540705 3415 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:27.540776 3415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:27.540992 3415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:27.543053 3415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "b2888eb3517247efb42e46ba8c00c9d2"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "4c6e8ef24de5b6fb5911fb11a168945d"
server_key_iv: "4a2a9ededa2e11fc9cdc55de3cf3080d"
server_key_version: "encryptionkey@0"
I20260806 04:37:27.543601 3415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "b2888eb3517247efb42e46ba8c00c9d2"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "4c6e8ef24de5b6fb5911fb11a168945d"
server_key_iv: "4a2a9ededa2e11fc9cdc55de3cf3080d"
server_key_version: "encryptionkey@0"
I20260806 04:37:27.547268 3415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:37:27.549705 3440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:27.550947 3415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:37:27.551085 3415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "b2888eb3517247efb42e46ba8c00c9d2"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "4c6e8ef24de5b6fb5911fb11a168945d"
server_key_iv: "4a2a9ededa2e11fc9cdc55de3cf3080d"
server_key_version: "encryptionkey@0"
I20260806 04:37:27.551188 3415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:27.572942 3415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:27.576059 3415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:27.576241 3415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:27.583210 3415 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:43745
I20260806 04:37:27.583520 3521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:43745 every 8 connection(s)
I20260806 04:37:27.584165 3415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:27.586947 3522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:27.589255 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 3415
I20260806 04:37:27.589382 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:37:27.589707 12351 external_mini_cluster.cc:1428] Setting key 6644a4d867cf9cd1733bd13b8b42be77
I20260806 04:37:27.593365 3522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap starting.
I20260806 04:37:27.595971 3522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:27.596822 3522 log.cc:824] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:27.598942 3522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: No bootstrap required, opened a new log
Aug 06 04:37:27 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:27.602814 3522 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:27.603085 3522 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:27.603202 3522 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2888eb3517247efb42e46ba8c00c9d2, State: Initialized, Role: FOLLOWER
I20260806 04:37:27.603641 3522 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:27.603770 3522 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:37:27.603835 3522 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:37:27.603932 3522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:27.604988 3522 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:27.605360 3522 leader_election.cc:302] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2888eb3517247efb42e46ba8c00c9d2; no voters:
I20260806 04:37:27.605700 3522 leader_election.cc:288] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:37:27.605789 3530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:27.605777 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:27.591171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.12.15.254:43745)
0806 04:37:27.591672 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:27.591691 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:27.591734 (+ 43us) server_negotiation.cc:407] Connection header received
0806 04:37:27.592568 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:27.592592 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:27.593000 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:27.593357 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:27.594049 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:27.595514 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:27.596470 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:27.597138 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:27.597298 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:27.599934 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:27.599940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:27.599956 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:27.599988 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:27.602493 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:27.602930 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:27.602936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:27.602941 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:27.603081 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:27.603376 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:27.603380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:27.603382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:27.603842 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:27.604051 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:27.604502 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 04:37:27.604820 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":150,"threads_started":1}
I20260806 04:37:27.606016 3530 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 LEADER]: Becoming Leader. State: Replica: b2888eb3517247efb42e46ba8c00c9d2, State: Running, Role: LEADER
I20260806 04:37:27.606350 3530 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:27.606941 3522 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:27.608947 3531 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2888eb3517247efb42e46ba8c00c9d2. Latest consensus state: current_term: 1 leader_uuid: "b2888eb3517247efb42e46ba8c00c9d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } } }
I20260806 04:37:27.609112 3531 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:27.609615 3533 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2888eb3517247efb42e46ba8c00c9d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } } }
I20260806 04:37:27.609705 3533 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: This master's current role is: LEADER
W20260806 04:37:27.611977 3561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:37:27.612046 3561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:37:27.612109 3559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:27.612821 3559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:27.618465 3559 catalog_manager.cc:1347] Generated new cluster ID: b8c14d6fc56149daa0fb186720ce8e5d
I20260806 04:37:27.618539 3559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:37:27.635484 3559 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:37:27.636785 3559 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:37:27.646023 3559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Generated new TSK 0
I20260806 04:37:27.646831 3559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:37:27.680199 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:43745
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:27.795079 3571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:27.795336 3571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:27.795379 3571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:27.799382 3571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:27.799436 3571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:27.799494 3571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:37:27.804663 3571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:27.805650 3571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:27.806656 3571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:27.816076 3582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:27.816205 3581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:27.816094 3584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:27.816174 3571 server_base.cc:1034] running on GCE node
I20260806 04:37:27.816756 3571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:27.817342 3571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:27.818514 3571 hybrid_clock.cc:648] HybridClock initialized: now 1785991047818493 us; error 29 us; skew 500 ppm
Aug 06 04:37:27 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991047, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:27.821156 3571 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:37:27.821973 3571 webserver.cc:460] Webserver started at http://127.12.15.193:38581/ using document root <none> and password file <none>
I20260806 04:37:27.822523 3571 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:27.822587 3571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:27.822789 3571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:27.825152 3571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "bf49e05e753a4ad9b018ea85d591d229"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "14ef70885810554ae4e673a6aba6a456"
server_key_iv: "a81e794405b5a760e2932130fc5fe9d3"
server_key_version: "encryptionkey@0"
I20260806 04:37:27.825760 3571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "bf49e05e753a4ad9b018ea85d591d229"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "14ef70885810554ae4e673a6aba6a456"
server_key_iv: "a81e794405b5a760e2932130fc5fe9d3"
server_key_version: "encryptionkey@0"
I20260806 04:37:27.829403 3571 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:37:27.831774 3593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:27.833081 3571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:37:27.833194 3571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "bf49e05e753a4ad9b018ea85d591d229"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "14ef70885810554ae4e673a6aba6a456"
server_key_iv: "a81e794405b5a760e2932130fc5fe9d3"
server_key_version: "encryptionkey@0"
I20260806 04:37:27.833292 3571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:27.860083 3571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:27.863296 3571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:27.863474 3571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:27.864068 3571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:27.864915 3571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:27.864972 3571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:27.865037 3571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:27.865062 3571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:27.873118 3571 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38231
I20260806 04:37:27.873150 3767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38231 every 8 connection(s)
I20260806 04:37:27.873955 3571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:37:27.877490 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 3571
I20260806 04:37:27.877621 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:37:27.877882 12351 external_mini_cluster.cc:1428] Setting key 3ec55aa2723a7f60cecc598c818c8e7c
Aug 06 04:37:27 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991047, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:27.887750 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:27.875702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:44169 (local address 127.12.15.254:43745)
0806 04:37:27.875840 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:27.875845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:27.876694 (+ 849us) server_negotiation.cc:407] Connection header received
0806 04:37:27.878062 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:27.878066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:27.878123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:27.878200 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:27.879435 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:27.879971 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:27.881177 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:27.881409 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:27.881495 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:27.884218 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:27.884223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:27.884226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:27.884262 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:27.885581 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:27.886194 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:27.886196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:27.886197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:27.886263 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:27.886607 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:27.886609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:27.886610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:27.886790 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:27.886862 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:27.887502 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 04:37:27.887610 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:37:27.888865 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:27.876062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:44169)
0806 04:37:27.876552 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:27.876596 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:27.877855 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:27.878369 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:27.878378 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:27.878767 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:27.879302 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:27.879317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:27.880090 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:27.880093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:27.881056 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:27.881069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:27.881553 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:27.881557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:27.881746 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:27.882309 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:27.882330 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:27.884036 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:27.885711 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:27.885719 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:27.885737 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:27.886102 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:27.886338 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:27.886341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:27.886343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:27.886527 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:27.886886 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:27.886892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:27.887229 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 04:37:27.887449 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 04:37:27.887815 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":115,"threads_started":1}
I20260806 04:37:27.890149 3768 heartbeater.cc:347] Connected to a master server at 127.12.15.254:43745
I20260806 04:37:27.890383 3768 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:27.890870 3768 heartbeater.cc:511] Master 127.12.15.254:43745 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:37:27.892643 3467 ts_manager.cc:194] Registered new tserver with Master: bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193:38231)
I20260806 04:37:27.893909 3467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:44169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:37:27.907297 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:43745
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:28.021368 3774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:28.021652 3774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:28.021697 3774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:28.025856 3774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:28.025919 3774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:28.025979 3774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:37:28.031306 3774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:28.032465 3774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:28.033517 3774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:28.043850 3782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:28.043852 3783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:28.043825 3788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:28.044263 3774 server_base.cc:1034] running on GCE node
I20260806 04:37:28.044724 3774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:28.045377 3774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:28.046556 3774 hybrid_clock.cc:648] HybridClock initialized: now 1785991048046529 us; error 38 us; skew 500 ppm
Aug 06 04:37:28 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:28.049327 3774 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:37:28.050295 3774 webserver.cc:460] Webserver started at http://127.12.15.194:35547/ using document root <none> and password file <none>
I20260806 04:37:28.050844 3774 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:28.050905 3774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:28.051083 3774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:28.053313 3774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "5708813ade1847c1a3285852a0059d3f"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "41b01c465aa8653df0eb0de9ff34c2e8"
server_key_iv: "843a808a9799b68643cae0a27bd0a69e"
server_key_version: "encryptionkey@0"
I20260806 04:37:28.053891 3774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "5708813ade1847c1a3285852a0059d3f"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "41b01c465aa8653df0eb0de9ff34c2e8"
server_key_iv: "843a808a9799b68643cae0a27bd0a69e"
server_key_version: "encryptionkey@0"
I20260806 04:37:28.057741 3774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:37:28.060142 3798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:28.061379 3774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:37:28.061486 3774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "5708813ade1847c1a3285852a0059d3f"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "41b01c465aa8653df0eb0de9ff34c2e8"
server_key_iv: "843a808a9799b68643cae0a27bd0a69e"
server_key_version: "encryptionkey@0"
I20260806 04:37:28.061587 3774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:28.089880 3774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:28.093108 3774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:28.093299 3774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:28.093954 3774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:28.094910 3774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:28.094959 3774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:28.095008 3774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:28.095046 3774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:28.103578 3774 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:44159
I20260806 04:37:28.103600 3992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:44159 every 8 connection(s)
I20260806 04:37:28.104555 3774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:37:28.104784 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 3774
I20260806 04:37:28.104907 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:37:28.105217 12351 external_mini_cluster.cc:1428] Setting key 6b9a366c70824f17dac127c3d51ee8c2
Aug 06 04:37:28 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:37:28.119292 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.106457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:49485 (local address 127.12.15.254:43745)
0806 04:37:28.106601 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.106606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.107561 (+ 955us) server_negotiation.cc:407] Connection header received
0806 04:37:28.109069 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.109072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.109132 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.109236 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:28.110508 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.111251 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.112423 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.112773 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.112894 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.115463 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:28.115467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:28.115470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:28.115502 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:28.117037 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:28.117583 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:28.117586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:28.117588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:28.117669 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:28.118040 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:28.118044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:28.118046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:28.118264 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:28.118356 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.118973 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.119112 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:37:28.120126 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.106890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:49485)
0806 04:37:28.107399 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.107451 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.108831 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.109428 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.109437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.109827 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:28.110367 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.110383 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.111378 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.111380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.112266 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.112275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.112934 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.112937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.113082 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.113685 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:28.113702 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:28.115279 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:28.117160 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:28.117166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:28.117184 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:28.117482 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:28.117755 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:28.117758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:28.117760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:28.117907 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:28.118367 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:28.118375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:28.118699 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.118886 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.119242 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":116,"threads_started":1}
I20260806 04:37:28.121520 3993 heartbeater.cc:347] Connected to a master server at 127.12.15.254:43745
I20260806 04:37:28.121791 3993 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:28.122321 3993 heartbeater.cc:511] Master 127.12.15.254:43745 requested a full tablet report, sending...
I20260806 04:37:28.123535 3467 ts_manager.cc:194] Registered new tserver with Master: 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159)
I20260806 04:37:28.124143 3467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:49485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:37:28.135828 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:43745
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:28.247064 3999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:28.247299 3999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:28.247340 3999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:28.251286 3999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:28.251335 3999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:28.251397 3999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:37:28.256548 3999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:28.257488 3999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:28.258467 3999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:28.267660 4017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:28.267772 3999 server_base.cc:1034] running on GCE node
W20260806 04:37:28.267745 4013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:28.267673 4024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:28.268280 3999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:28.268901 3999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:28.270076 3999 hybrid_clock.cc:648] HybridClock initialized: now 1785991048270054 us; error 32 us; skew 500 ppm
Aug 06 04:37:28 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:28.272781 3999 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:37:28.273610 3999 webserver.cc:460] Webserver started at http://127.12.15.195:46523/ using document root <none> and password file <none>
I20260806 04:37:28.274150 3999 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:28.274216 3999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:28.274425 3999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:37:28.276438 3999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "5fa917b8ccac4f5f832d0312ae717791"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "2a525107082881d604efb1dec3961732"
server_key_iv: "d974b0168c4c73ace191c658ef98df0e"
server_key_version: "encryptionkey@0"
I20260806 04:37:28.276983 3999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "5fa917b8ccac4f5f832d0312ae717791"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "2a525107082881d604efb1dec3961732"
server_key_iv: "d974b0168c4c73ace191c658ef98df0e"
server_key_version: "encryptionkey@0"
I20260806 04:37:28.280616 3999 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:37:28.283030 4033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:28.284302 3999 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:28.284446 3999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5fa917b8ccac4f5f832d0312ae717791"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "2a525107082881d604efb1dec3961732"
server_key_iv: "d974b0168c4c73ace191c658ef98df0e"
server_key_version: "encryptionkey@0"
I20260806 04:37:28.284548 3999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:28.303093 3999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:28.305841 3999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:28.306022 3999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:28.306623 3999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:28.307462 3999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:28.307510 3999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:28.307564 3999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:28.307595 3999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:28.315806 3999 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:41069
I20260806 04:37:28.315881 4201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:41069 every 8 connection(s)
I20260806 04:37:28.316704 3999 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:37:28.322686 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 3999
I20260806 04:37:28.322818 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:37:28.323081 12351 external_mini_cluster.cc:1428] Setting key 00787b2d2202abfc2ec59bf4e9bc3d18
Aug 06 04:37:28 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991048, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:28.329905 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.318476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:56121 (local address 127.12.15.254:43745)
0806 04:37:28.318604 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.318607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.319377 (+ 770us) server_negotiation.cc:407] Connection header received
0806 04:37:28.320806 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.320809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.320872 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.320948 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:28.322105 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.322653 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.323854 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.324164 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.324264 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.326797 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:28.326801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:28.326804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:28.326830 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:28.327945 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:28.328429 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:28.328433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:28.328435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:28.328513 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:28.328849 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:28.328853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:28.328855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:28.329034 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:28.329118 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.329608 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.329741 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:37:28.330747 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.318742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:56121)
0806 04:37:28.319211 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.319260 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.320586 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.321134 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.321141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.321489 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:28.321954 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.321967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.322772 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.322775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.323732 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.323745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.324331 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.324356 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.324587 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.325206 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:28.325228 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:28.326640 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:28.328049 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:28.328055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:28.328070 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:28.328325 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:28.328594 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:28.328597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:28.328599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:28.328752 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:28.329138 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:28.329145 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:28.329377 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.329532 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.329854 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260806 04:37:28.331801 4203 heartbeater.cc:347] Connected to a master server at 127.12.15.254:43745
I20260806 04:37:28.332052 4203 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:28.332574 4203 heartbeater.cc:511] Master 127.12.15.254:43745 requested a full tablet report, sending...
I20260806 04:37:28.333611 3467 ts_manager.cc:194] Registered new tserver with Master: 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
I20260806 04:37:28.334198 3467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:56121
I20260806 04:37:28.336992 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:37:28.346196 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.338992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.12.15.254:43745)
0806 04:37:28.339127 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.339131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.339294 (+ 163us) server_negotiation.cc:407] Connection header received
0806 04:37:28.339389 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.339392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.339429 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.339494 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:28.340240 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.340845 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.341688 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.341909 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.341988 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.343225 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:28.343229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:28.343230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:28.343254 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:28.344578 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:28.344957 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:28.344960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:28.344961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:28.345044 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:28.345399 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:28.345402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:28.345403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:28.345644 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:28.345751 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.345971 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.346070 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:37:28.349195 3467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:37:28.352312 3467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:37:28.367475 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.361962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38231 (local address 127.0.0.1:46626)
0806 04:37:28.362955 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.362968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.363064 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.363449 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.363452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.363470 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:28.363549 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.363556 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.364528 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.364532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.366057 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.366069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.366927 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.366930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.367042 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.367080 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.367188 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.367307 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":40,"thread_start_us":100,"threads_started":1}
I20260806 04:37:28.367614 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.362386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.12.15.193:38231)
0806 04:37:28.362815 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.362832 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.363067 (+ 235us) server_negotiation.cc:407] Connection header received
0806 04:37:28.363152 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.363158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.363324 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.363451 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:28.363637 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.364394 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.366195 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.366814 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.366914 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.367073 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.367347 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.367454 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":118,"threads_started":1}
I20260806 04:37:28.367486 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.361157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44159 (local address 127.0.0.1:54540)
0806 04:37:28.361543 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.361602 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.361740 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.362835 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.362841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.362868 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:28.363018 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.363026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.364644 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.364647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.366075 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.366082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.367148 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.367152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.367231 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.367251 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.367319 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.367375 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":232,"thread_start_us":95,"threads_started":1}
I20260806 04:37:28.367702 4216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.361398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41069 (local address 127.0.0.1:34340)
0806 04:37:28.361984 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.361997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.362082 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.363003 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.363005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.363023 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:28.363130 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.363136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.364808 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.364812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.366041 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.366051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.367319 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.367323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.367409 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.367431 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.367497 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.367575 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":102,"spinlock_wait_cycles":3968,"thread_start_us":88,"threads_started":1}
I20260806 04:37:28.367839 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.361319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54540 (local address 127.12.15.194:44159)
0806 04:37:28.362326 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.362338 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.362363 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:37:28.362457 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.362462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.362666 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.362826 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:28.363155 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.364513 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.366189 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.366999 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.367139 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.367255 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.367595 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.367702 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":186,"threads_started":1}
I20260806 04:37:28.369024 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.361928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.12.15.195:41069)
0806 04:37:28.362355 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.362365 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.362383 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:37:28.362467 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.362471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.362666 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.362826 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:28.363256 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.364549 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.366158 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.367000 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.367113 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.367255 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.367800 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.367903 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":154,"threads_started":1}
I20260806 04:37:28.371189 4104 tablet_service.cc:1467] Processing CreateTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e (DEFAULT_TABLE table=test-table [id=4c21603d9e8b4cedaa430ef866f73094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:28.371155 3878 tablet_service.cc:1467] Processing CreateTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e (DEFAULT_TABLE table=test-table [id=4c21603d9e8b4cedaa430ef866f73094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:28.371150 3667 tablet_service.cc:1467] Processing CreateTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e (DEFAULT_TABLE table=test-table [id=4c21603d9e8b4cedaa430ef866f73094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:28.372227 3878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0666700efcf2499aa5aca1af7ebf9f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:28.372227 3667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0666700efcf2499aa5aca1af7ebf9f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:28.372241 4104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0666700efcf2499aa5aca1af7ebf9f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:28.378696 4227 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Bootstrap starting.
I20260806 04:37:28.380298 4229 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Bootstrap starting.
I20260806 04:37:28.380646 4228 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Bootstrap starting.
I20260806 04:37:28.381366 4227 tablet_bootstrap.cc:654] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:28.382432 4229 tablet_bootstrap.cc:654] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:28.382756 4228 tablet_bootstrap.cc:654] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:28.383137 4227 log.cc:824] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:28.383490 4228 log.cc:824] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:28.383706 4229 log.cc:824] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:28.385087 4227 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: No bootstrap required, opened a new log
I20260806 04:37:28.385273 4227 ts_tablet_manager.cc:1397] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 04:37:28.385437 4229 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: No bootstrap required, opened a new log
I20260806 04:37:28.385437 4228 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: No bootstrap required, opened a new log
I20260806 04:37:28.385627 4228 ts_tablet_manager.cc:1397] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:37:28.385627 4229 ts_tablet_manager.cc:1397] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:37:28.388254 4227 raft_consensus.cc:348] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.388557 4227 raft_consensus.cc:374] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:28.388644 4227 raft_consensus.cc:729] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa917b8ccac4f5f832d0312ae717791, State: Initialized, Role: FOLLOWER
I20260806 04:37:28.389076 4227 consensus_queue.cc:260] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.389539 4228 raft_consensus.cc:348] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.389832 4228 raft_consensus.cc:374] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:28.389910 4228 raft_consensus.cc:729] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5708813ade1847c1a3285852a0059d3f, State: Initialized, Role: FOLLOWER
I20260806 04:37:28.389357 4229 raft_consensus.cc:348] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.390093 4229 raft_consensus.cc:374] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:28.390187 4229 raft_consensus.cc:729] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf49e05e753a4ad9b018ea85d591d229, State: Initialized, Role: FOLLOWER
I20260806 04:37:28.390249 4227 ts_tablet_manager.cc:1428] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 04:37:28.390342 4228 consensus_queue.cc:260] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.390584 4229 consensus_queue.cc:260] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.390656 4203 heartbeater.cc:503] Master 127.12.15.254:43745 was elected leader, sending a full tablet report...
I20260806 04:37:28.390961 3993 heartbeater.cc:503] Master 127.12.15.254:43745 was elected leader, sending a full tablet report...
I20260806 04:37:28.391279 4228 ts_tablet_manager.cc:1428] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:37:28.391456 4229 ts_tablet_manager.cc:1428] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:37:28.391742 3768 heartbeater.cc:503] Master 127.12.15.254:43745 was elected leader, sending a full tablet report...
I20260806 04:37:28.448001 4235 raft_consensus.cc:482] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:28.448226 4235 raft_consensus.cc:504] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.449456 4235 leader_election.cc:288] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
I20260806 04:37:28.453408 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.449843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:59063 (local address 127.12.15.195:41069)
0806 04:37:28.449977 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.449981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.450211 (+ 230us) server_negotiation.cc:407] Connection header received
0806 04:37:28.450353 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.450357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.450403 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.450466 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:28.450838 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.451319 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.452467 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.452858 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.452967 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.453035 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.453206 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.453282 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 04:37:28.453429 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.449802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:49351 (local address 127.12.15.194:44159)
0806 04:37:28.449926 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.449931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.450235 (+ 304us) server_negotiation.cc:407] Connection header received
0806 04:37:28.450370 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.450372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.450417 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.450480 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:28.450803 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.451317 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.452474 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.452869 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.452973 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.453028 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.453239 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.453308 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 04:37:28.453449 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.449759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44159 (local address 127.12.15.193:49351)
0806 04:37:28.450132 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.450145 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.450265 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.450495 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.450498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.450544 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:28.450691 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.450696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.451410 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.451414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.452363 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.452375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.452973 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.452977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.453132 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.453151 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.453213 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.453285 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":105,"spinlock_wait_cycles":22400,"thread_start_us":140,"threads_started":1}
I20260806 04:37:28.453435 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.449748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41069 (local address 127.12.15.193:59063)
0806 04:37:28.450106 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.450121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:28.450244 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:28.450490 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:28.450492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:28.450541 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:28.450705 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.450710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.451426 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.451429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.452324 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:28.452365 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.452967 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.452971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:28.453070 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.453112 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 04:37:28.453178 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 04:37:28.453255 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":62,"spinlock_wait_cycles":29824,"thread_start_us":158,"threads_started":1}
I20260806 04:37:28.454196 4134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0666700efcf2499aa5aca1af7ebf9f1e" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa917b8ccac4f5f832d0312ae717791" is_pre_election: true
I20260806 04:37:28.454196 3916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0666700efcf2499aa5aca1af7ebf9f1e" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5708813ade1847c1a3285852a0059d3f" is_pre_election: true
I20260806 04:37:28.454545 3916 raft_consensus.cc:2454] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 0.
I20260806 04:37:28.454545 4134 raft_consensus.cc:2454] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 0.
I20260806 04:37:28.455071 3598 leader_election.cc:302] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5708813ade1847c1a3285852a0059d3f, bf49e05e753a4ad9b018ea85d591d229; no voters:
I20260806 04:37:28.455380 4235 raft_consensus.cc:2781] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:37:28.455482 4235 raft_consensus.cc:482] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:37:28.455543 4235 raft_consensus.cc:3037] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:28.456830 4235 raft_consensus.cc:504] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.457150 4235 leader_election.cc:288] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 election: Requested vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
I20260806 04:37:28.457473 3916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0666700efcf2499aa5aca1af7ebf9f1e" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5708813ade1847c1a3285852a0059d3f"
I20260806 04:37:28.457484 4134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0666700efcf2499aa5aca1af7ebf9f1e" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa917b8ccac4f5f832d0312ae717791"
I20260806 04:37:28.457605 3916 raft_consensus.cc:3037] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:28.457605 4134 raft_consensus.cc:3037] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:28.458865 4134 raft_consensus.cc:2454] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 1.
I20260806 04:37:28.458865 3916 raft_consensus.cc:2454] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 1.
I20260806 04:37:28.459187 3597 leader_election.cc:302] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa917b8ccac4f5f832d0312ae717791, bf49e05e753a4ad9b018ea85d591d229; no voters:
I20260806 04:37:28.459378 4235 raft_consensus.cc:2781] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:28.459616 4235 raft_consensus.cc:686] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 LEADER]: Becoming Leader. State: Replica: bf49e05e753a4ad9b018ea85d591d229, State: Running, Role: LEADER
I20260806 04:37:28.459964 4235 consensus_queue.cc:237] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:28.463171 3467 catalog_manager.cc:5495] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "bf49e05e753a4ad9b018ea85d591d229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260806 04:37:28.491055 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.486424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.12.15.193:38231)
0806 04:37:28.486570 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:28.486574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:28.486588 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:37:28.486668 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:28.486671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:28.486760 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:28.486869 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:28.487192 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.487966 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.489229 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:28.489574 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:28.489697 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:28.490024 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:28.490645 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:37:28.490780 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:28.490820 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 04:37:28.490894 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:37:28.499593 3916 raft_consensus.cc:1261] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Refusing update from remote peer bf49e05e753a4ad9b018ea85d591d229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:28.499598 4134 raft_consensus.cc:1261] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Refusing update from remote peer bf49e05e753a4ad9b018ea85d591d229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:28.500301 4240 consensus_queue.cc:1035] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:28.500525 4235 consensus_queue.cc:1035] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:28.510640 4249 mvcc.cc:205] Tried to move back new op lower bound from 7315419334645571584 to 7315419334494912512. Current Snapshot: MvccSnapshot[applied={T|T < 7315419334645571584}]
I20260806 04:37:28.510946 4250 mvcc.cc:205] Tried to move back new op lower bound from 7315419334645571584 to 7315419334494912512. Current Snapshot: MvccSnapshot[applied={T|T < 7315419334645571584}]
I20260806 04:37:28.518784 3466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:28.518973 3466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:28.521771 3466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Sending DeleteTablet for 3 replicas of tablet 0666700efcf2499aa5aca1af7ebf9f1e
I20260806 04:37:28.522727 3878 tablet_service.cc:1514] Processing DeleteTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:28 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:54540
I20260806 04:37:28.522948 4104 tablet_service.cc:1514] Processing DeleteTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:28 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:34340
I20260806 04:37:28.523272 4266 tablet_replica.cc:330] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: stopping tablet replica
I20260806 04:37:28.523379 3667 tablet_service.cc:1514] Processing DeleteTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:28 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:46626
I20260806 04:37:28.523571 4266 raft_consensus.cc:2229] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:28.523855 4268 tablet_replica.cc:330] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: stopping tablet replica
I20260806 04:37:28.523886 4266 raft_consensus.cc:2258] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:28.524165 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 3571
I20260806 04:37:28.525864 4266 ts_tablet_manager.cc:1905] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:28.528311 4266 ts_tablet_manager.cc:1918] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:28.528419 4266 log.cc:1196] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/0666700efcf2499aa5aca1af7ebf9f1e
I20260806 04:37:28.528713 4266 ts_tablet_manager.cc:1939] T 0666700efcf2499aa5aca1af7ebf9f1e P 5708813ade1847c1a3285852a0059d3f: Deleting consensus metadata
I20260806 04:37:28.530007 3446 catalog_manager.cc:4841] TS 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): tablet 0666700efcf2499aa5aca1af7ebf9f1e (table test-table [id=4c21603d9e8b4cedaa430ef866f73094]) successfully deleted
I20260806 04:37:28.532835 4267 tablet_replica.cc:330] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: stopping tablet replica
I20260806 04:37:28.533152 4267 raft_consensus.cc:2229] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:28.533468 4267 raft_consensus.cc:2258] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:28.534879 4267 ts_tablet_manager.cc:1905] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:28.537065 4267 ts_tablet_manager.cc:1918] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:28.537164 4267 log.cc:1196] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/0666700efcf2499aa5aca1af7ebf9f1e
I20260806 04:37:28.537447 4267 ts_tablet_manager.cc:1939] T 0666700efcf2499aa5aca1af7ebf9f1e P 5fa917b8ccac4f5f832d0312ae717791: Deleting consensus metadata
I20260806 04:37:28.538657 3444 catalog_manager.cc:4841] TS 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): tablet 0666700efcf2499aa5aca1af7ebf9f1e (table test-table [id=4c21603d9e8b4cedaa430ef866f73094]) successfully deleted
W20260806 04:37:28.539989 3446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:38231 (error 108)
I20260806 04:37:28.540850 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 3774
I20260806 04:37:28.546129 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:28.540956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38231 (local address 127.0.0.1:46646)
0806 04:37:28.545929 (+ 4973us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:28.546029 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4856}
W20260806 04:37:28.546340 3446 catalog_manager.cc:4584] TS bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193:38231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0666700efcf2499aa5aca1af7ebf9f1e: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38231: connect: Connection refused (error 111)
I20260806 04:37:28.553503 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 3999
I20260806 04:37:28.567167 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 3415
I20260806 04:37:48.581863 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43745
--webserver_interface=127.12.15.254
--webserver_port=37593
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:43745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:37:48.711539 4271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:48.711817 4271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:48.711858 4271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:48.715847 4271 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 04:37:48.715902 4271 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 04:37:48.715917 4271 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 04:37:48.715929 4271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:37:48.715946 4271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:48.715965 4271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:48.715994 4271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:37:48.716028 4271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:37:48.721365 4271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:43745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=37593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:48.722477 4271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:48.723533 4271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:48.737666 4281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:48.737653 4278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:48.737689 4279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:48.737721 4271 server_base.cc:1034] running on GCE node
I20260806 04:37:48.738605 4271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:48.739795 4271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:48.740967 4271 hybrid_clock.cc:648] HybridClock initialized: now 1785991068740946 us; error 33 us; skew 500 ppm
Aug 06 04:37:48 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991068, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:48.743801 4271 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:37:48.744771 4271 webserver.cc:460] Webserver started at http://127.12.15.254:37593/ using document root <none> and password file <none>
I20260806 04:37:48.745353 4271 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:48.745438 4271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:48.749411 4271 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:37:48.751637 4292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:48.753012 4271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:48.753137 4271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "b2888eb3517247efb42e46ba8c00c9d2"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "4c6e8ef24de5b6fb5911fb11a168945d"
server_key_iv: "4a2a9ededa2e11fc9cdc55de3cf3080d"
server_key_version: "encryptionkey@0"
I20260806 04:37:48.753768 4271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:48.766639 4271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:48.769556 4271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:48.769728 4271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:48.776647 4271 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:43745
I20260806 04:37:48.776969 4372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:43745 every 8 connection(s)
I20260806 04:37:48.777737 4271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:37:48.782150 4373 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:37:48.783829 4373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap starting.
I20260806 04:37:48.784010 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 4271
I20260806 04:37:48.784574 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:38231
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:43745
--webserver_port=38581
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:48.803190 4373 log.cc:824] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:48.813532 4373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:37:48.813923 4373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap complete.
I20260806 04:37:48.817951 4373 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:48.818347 4373 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2888eb3517247efb42e46ba8c00c9d2, State: Initialized, Role: FOLLOWER
I20260806 04:37:48.818869 4373 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:48.818990 4373 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:37:48.819062 4373 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:37:48.819155 4373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 1 FOLLOWER]: Advancing to term 2
I20260806 04:37:48.821179 4373 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:48.821506 4373 leader_election.cc:302] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2888eb3517247efb42e46ba8c00c9d2; no voters:
I20260806 04:37:48.821810 4373 leader_election.cc:288] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 04:37:48.821949 4379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 04:37:48.822235 4379 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 LEADER]: Becoming Leader. State: Replica: b2888eb3517247efb42e46ba8c00c9d2, State: Running, Role: LEADER
I20260806 04:37:48.822573 4379 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:37:48.823019 4373 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:37:48.823645 4379 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2888eb3517247efb42e46ba8c00c9d2. Latest consensus state: current_term: 2 leader_uuid: "b2888eb3517247efb42e46ba8c00c9d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } } }
I20260806 04:37:48.824363 4379 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:48.824227 4380 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b2888eb3517247efb42e46ba8c00c9d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } } }
I20260806 04:37:48.825039 4380 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:37:48.827709 4402 catalog_manager.cc:1259] Loaded cluster ID: b8c14d6fc56149daa0fb186720ce8e5d
I20260806 04:37:48.827760 4402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: loading cluster ID for follower catalog manager: success
I20260806 04:37:48.829587 4402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: acquiring CA information for follower catalog manager: success
I20260806 04:37:48.830801 4402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:37:48.830924 4393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:37:48.833323 4393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:37:48.833804 4393 catalog_manager.cc:1259] Loaded cluster ID: b8c14d6fc56149daa0fb186720ce8e5d
I20260806 04:37:48.833855 4393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:37:48.834419 4393 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:37:48.834827 4393 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Loaded TSK: 0
I20260806 04:37:48.846470 4393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Generated new TSK 1
I20260806 04:37:48.846755 4393 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:37:48.913661 4375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:48.913924 4375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:48.913966 4375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:48.918335 4375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:48.918414 4375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:48.918475 4375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:37:48.924165 4375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:38231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=38581
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:48.925302 4375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:48.926317 4375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:48.937597 4414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:48.937623 4416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:48.937626 4413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:48.938047 4375 server_base.cc:1034] running on GCE node
I20260806 04:37:48.938591 4375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:48.939303 4375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:48.940459 4375 hybrid_clock.cc:648] HybridClock initialized: now 1785991068940427 us; error 54 us; skew 500 ppm
Aug 06 04:37:48 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991068, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:48.943452 4375 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:37:48.944736 4375 webserver.cc:460] Webserver started at http://127.12.15.193:38581/ using document root <none> and password file <none>
I20260806 04:37:48.945479 4375 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:48.945600 4375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:48.951066 4375 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:37:48.953881 4426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:48.955443 4375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:37:48.955571 4375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "bf49e05e753a4ad9b018ea85d591d229"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "14ef70885810554ae4e673a6aba6a456"
server_key_iv: "a81e794405b5a760e2932130fc5fe9d3"
server_key_version: "encryptionkey@0"
I20260806 04:37:48.956282 4375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:48.979744 4375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:48.983407 4375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:48.983572 4375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:48.984138 4375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:48.985277 4437 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 04:37:48.989504 4375 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 04:37:48.989578 4375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 04:37:48.989639 4375 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 04:37:48.991443 4375 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 04:37:48.991492 4375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 04:37:48.991837 4437 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Bootstrap starting.
I20260806 04:37:49.001098 4375 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38231
I20260806 04:37:49.001154 4620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38231 every 8 connection(s)
I20260806 04:37:49.002048 4375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:37:49.002596 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 4375
I20260806 04:37:49.003170 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:44159
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:43745
--webserver_port=35547
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:37:49.011588 4437 log.cc:824] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Log is configured to *not* fsync() on all Append() calls
Aug 06 04:37:49 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991068, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:49.019361 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.004277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:59815 (local address 127.12.15.254:43745)
0806 04:37:49.004852 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.004872 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.005323 (+ 451us) server_negotiation.cc:407] Connection header received
0806 04:37:49.006473 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.006484 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.006784 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.007105 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:49.008280 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.009333 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.010534 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.011008 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.011179 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.013831 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:49.013837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:49.013851 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:49.013885 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:49.015565 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.016111 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.016117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.016121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.016297 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.016850 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.016854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.016856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.017359 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:49.017515 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.018150 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.018437 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":143,"threads_started":1}
I20260806 04:37:49.019379 4625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.004287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:59815)
0806 04:37:49.005146 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.005201 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.006050 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.007178 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.007187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.007624 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:49.008100 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.008116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.009531 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.009534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.010406 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.010417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.011157 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.011161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.011355 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.011952 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:49.011970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:49.013618 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:49.015690 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.015697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.015712 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.015991 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.016504 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.016507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.016509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.016706 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.017576 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:49.017583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:49.017820 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.017997 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.018380 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":148,"threads_started":1}
I20260806 04:37:49.021924 4437 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:37:49.022454 4437 tablet_bootstrap.cc:492] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Bootstrap complete.
I20260806 04:37:49.022677 4622 heartbeater.cc:347] Connected to a master server at 127.12.15.254:43745
I20260806 04:37:49.022922 4622 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:49.023479 4622 heartbeater.cc:511] Master 127.12.15.254:43745 requested a full tablet report, sending...
I20260806 04:37:49.023870 4437 ts_tablet_manager.cc:1397] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.008s
I20260806 04:37:49.026181 4319 ts_manager.cc:194] Registered new tserver with Master: bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193:38231)
I20260806 04:37:49.027048 4437 raft_consensus.cc:348] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:49.027597 4437 raft_consensus.cc:729] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf49e05e753a4ad9b018ea85d591d229, State: Initialized, Role: FOLLOWER
I20260806 04:37:49.028113 4437 consensus_queue.cc:260] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } }
I20260806 04:37:49.029191 4437 ts_tablet_manager.cc:1428] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
I20260806 04:37:49.030088 4319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:59815
I20260806 04:37:49.032670 4622 heartbeater.cc:503] Master 127.12.15.254:43745 was elected leader, sending a full tablet report...
Aug 06 04:37:49 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991068, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.193@KRBTEST.COM
I20260806 04:37:49.041731 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.029350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38231 (local address 127.0.0.1:54372)
0806 04:37:49.030454 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.030507 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.030665 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.031219 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.031225 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.031680 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:49.031823 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.031832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.033097 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.033100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.033971 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.033984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.034732 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.034735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.034816 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.035390 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:49.035403 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:49.037209 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:49.039597 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.039616 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.039621 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.040007 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.040412 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.040415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.040417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.040512 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.041127 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:49.041133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:49.041254 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.041382 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.041586 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":118,"threads_started":1}
I20260806 04:37:49.041821 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.029741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.12.15.193:38231)
0806 04:37:49.030118 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.030130 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.030619 (+ 489us) server_negotiation.cc:407] Connection header received
0806 04:37:49.030877 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.030883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.031047 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.031178 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:49.031960 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.032983 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.034101 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.034608 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.034719 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.037368 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:49.037380 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:49.037394 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:49.037425 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:49.039292 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.040140 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.040147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.040148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.040250 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.040633 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.040636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.040638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.040990 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:49.041126 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.041471 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.041651 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":94,"threads_started":1}
I20260806 04:37:49.042920 4508 tablet_service.cc:1514] Processing DeleteTablet for tablet 0666700efcf2499aa5aca1af7ebf9f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:28 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:54372
I20260806 04:37:49.043977 4635 tablet_replica.cc:330] stopping tablet replica
I20260806 04:37:49.044292 4635 raft_consensus.cc:2229] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:49.044598 4635 raft_consensus.cc:2258] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:49.045805 4635 ts_tablet_manager.cc:1905] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:49.049433 4635 ts_tablet_manager.cc:1918] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:49.049530 4635 log.cc:1196] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal/wals/0666700efcf2499aa5aca1af7ebf9f1e
I20260806 04:37:49.049757 4635 ts_tablet_manager.cc:1939] T 0666700efcf2499aa5aca1af7ebf9f1e P bf49e05e753a4ad9b018ea85d591d229: Deleting consensus metadata
I20260806 04:37:49.051584 4296 catalog_manager.cc:4841] TS bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193:38231): tablet 0666700efcf2499aa5aca1af7ebf9f1e (table test-table [id=4c21603d9e8b4cedaa430ef866f73094]) successfully deleted
W20260806 04:37:49.132126 4628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:49.132416 4628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:49.132472 4628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:49.136601 4628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:49.136659 4628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:49.136722 4628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:37:49.141964 4628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:44159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=35547
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:49.143008 4628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:49.143991 4628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:49.154026 4644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:49.154052 4649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:49.154034 4646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:49.154356 4628 server_base.cc:1034] running on GCE node
I20260806 04:37:49.154755 4628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:49.155390 4628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:49.156585 4628 hybrid_clock.cc:648] HybridClock initialized: now 1785991069156560 us; error 38 us; skew 500 ppm
Aug 06 04:37:49 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:49.159413 4628 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:37:49.160585 4628 webserver.cc:460] Webserver started at http://127.12.15.194:35547/ using document root <none> and password file <none>
I20260806 04:37:49.161211 4628 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:49.161307 4628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:49.165331 4628 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 04:37:49.167389 4660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:49.168725 4628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:49.168843 4628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "5708813ade1847c1a3285852a0059d3f"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "41b01c465aa8653df0eb0de9ff34c2e8"
server_key_iv: "843a808a9799b68643cae0a27bd0a69e"
server_key_version: "encryptionkey@0"
I20260806 04:37:49.169479 4628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:49.181783 4628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:49.184787 4628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:49.184969 4628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:49.185564 4628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:49.186405 4628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:49.186453 4628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:49.186508 4628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:49.186539 4628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:49.195384 4628 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:44159
I20260806 04:37:49.195449 4844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:44159 every 8 connection(s)
I20260806 04:37:49.196434 4628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:37:49.202947 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 4628
I20260806 04:37:49.203388 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:41069
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:43745
--webserver_port=46523
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 04:37:49 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:49.210070 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.198299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:47095 (local address 127.12.15.254:43745)
0806 04:37:49.198443 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.198447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.199262 (+ 815us) server_negotiation.cc:407] Connection header received
0806 04:37:49.200632 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.200635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.200691 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.200764 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:49.201907 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.202436 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.203619 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.203849 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.203936 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.206563 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:49.206566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:49.206568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:49.206594 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:49.207864 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.208481 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.208484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.208485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.208584 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.208948 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.208950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.208951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.209117 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:49.209186 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.209820 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.209937 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:37:49.211169 4848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.198663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:47095)
0806 04:37:49.199116 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.199157 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.200434 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.200946 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.200952 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.201308 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:49.201769 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.201781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.202554 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.202558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.203500 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.203513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.203984 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.203988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.204151 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.204704 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:49.204720 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:49.206386 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:49.207983 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.207990 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.208004 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.208383 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.208672 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.208676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.208677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.208862 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.209207 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:49.209213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:49.209544 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.209761 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.210122 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260806 04:37:49.212390 4845 heartbeater.cc:347] Connected to a master server at 127.12.15.254:43745
I20260806 04:37:49.212664 4845 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:49.213296 4845 heartbeater.cc:511] Master 127.12.15.254:43745 requested a full tablet report, sending...
I20260806 04:37:49.214545 4319 ts_manager.cc:194] Registered new tserver with Master: 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159)
I20260806 04:37:49.215134 4319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:47095
W20260806 04:37:49.319567 4851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:37:49.319810 4851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:37:49.319852 4851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:37:49.323876 4851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:37:49.323936 4851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:37:49.324023 4851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:37:49.329617 4851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:41069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=46523
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:37:49.330598 4851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:37:49.331564 4851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:37:49.341233 4858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:49.341291 4861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:37:49.341274 4859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:37:49.341436 4851 server_base.cc:1034] running on GCE node
I20260806 04:37:49.341852 4851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:37:49.342483 4851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:37:49.343637 4851 hybrid_clock.cc:648] HybridClock initialized: now 1785991069343621 us; error 33 us; skew 500 ppm
Aug 06 04:37:49 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:37:49.346297 4851 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:37:49.347131 4851 webserver.cc:460] Webserver started at http://127.12.15.195:46523/ using document root <none> and password file <none>
I20260806 04:37:49.347671 4851 fs_manager.cc:359] Metadata directory not provided
I20260806 04:37:49.347757 4851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:37:49.351585 4851 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:37:49.353626 4872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:49.354816 4851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:37:49.354924 4851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5fa917b8ccac4f5f832d0312ae717791"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "2a525107082881d604efb1dec3961732"
server_key_iv: "d974b0168c4c73ace191c658ef98df0e"
server_key_version: "encryptionkey@0"
I20260806 04:37:49.355428 4851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:37:49.370720 4851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:37:49.373859 4851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:37:49.374043 4851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:37:49.374624 4851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:37:49.375465 4851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:37:49.375514 4851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:49.375559 4851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:37:49.375587 4851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:37:49.384038 4851 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:41069
I20260806 04:37:49.384063 5043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:41069 every 8 connection(s)
I20260806 04:37:49.385049 4851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:37:49.391181 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 4851
Aug 06 04:37:49 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991069, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:37:49.399264 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.386821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:40779 (local address 127.12.15.254:43745)
0806 04:37:49.386951 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.386955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.387769 (+ 814us) server_negotiation.cc:407] Connection header received
0806 04:37:49.389112 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.389115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.389173 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.389247 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:49.390407 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.390945 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.392309 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.392601 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.392690 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.395200 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:49.395204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:49.395207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:49.395235 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:49.396917 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.397486 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.397490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.397492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.397583 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.398164 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.398168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.398169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.398368 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:49.398453 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.398974 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.399094 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48,"spinlock_wait_cycles":20736}
I20260806 04:37:49.400178 5047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.387170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:40779)
0806 04:37:49.387626 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.387666 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.388896 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.389426 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.389433 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.389791 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:37:49.390267 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.390280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.391088 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.391091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.392187 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.392197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.392754 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.392759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.392913 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.393461 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:37:49.393477 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:37:49.395027 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:37:49.397018 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.397024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.397040 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.397369 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.397694 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:37:49.397911 (+ 217us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:37:49.397917 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:37:49.398065 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:37:49.398477 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:37:49.398484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:37:49.398735 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.398904 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.399237 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260806 04:37:49.400200 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.391814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.12.15.254:43745)
0806 04:37:49.392056 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.392060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.392145 (+ 85us) server_negotiation.cc:407] Connection header received
0806 04:37:49.392244 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.392247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.392301 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.392409 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:49.393129 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.393654 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.394759 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.395017 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.395121 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.396838 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:49.396842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:49.396845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:49.396903 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:49.398417 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.398845 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.398849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.398851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.398926 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.399176 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.399180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.399181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.399361 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:49.399559 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.399919 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.400043 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"spinlock_wait_cycles":6912,"thread_start_us":67,"threads_started":1}
I20260806 04:37:49.401363 5044 heartbeater.cc:347] Connected to a master server at 127.12.15.254:43745
I20260806 04:37:49.401593 5044 heartbeater.cc:464] Registering TS with master...
I20260806 04:37:49.402122 5044 heartbeater.cc:511] Master 127.12.15.254:43745 requested a full tablet report, sending...
I20260806 04:37:49.403118 4319 ts_manager.cc:194] Registered new tserver with Master: 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
I20260806 04:37:49.403656 4319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:40779
I20260806 04:37:49.413741 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:37:49.417737 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.414346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.12.15.254:43745)
0806 04:37:49.414467 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.414471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.414660 (+ 189us) server_negotiation.cc:407] Connection header received
0806 04:37:49.414845 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.414848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.414952 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.415049 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:49.415287 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.415784 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.416715 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.416944 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.417019 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.417293 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:49.417431 (+ 138us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 04:37:49.417637 (+ 206us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55206: token expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 04:37:49.417793 5050 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55206: token expired
W20260806 04:37:49.418112 4209 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.15.254:43745)
I20260806 04:37:49.424563 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.418416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.12.15.254:43745)
0806 04:37:49.418508 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.418511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.418536 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:37:49.418625 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.418627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.418661 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.418723 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:37:49.419240 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.419695 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.420419 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.420631 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.420706 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.421856 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:37:49.421859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:37:49.421861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:37:49.421884 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:37:49.423187 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.423553 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.423555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.423556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.423633 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:37:49.423814 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:37:49.423816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:37:49.423817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:37:49.424027 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:37:49.424164 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.424327 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.424449 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 04:37:49.427209 4319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:37:49.430696 4319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:37:49.439047 4508 tablet_service.cc:1467] Processing CreateTablet for tablet 82c3916229f049f891d25a9dd4627b57 (DEFAULT_TABLE table=test-table [id=eb0a5a1a8c5e4288bf07d58a06f1bef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:49.439747 4508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c3916229f049f891d25a9dd4627b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:49.442687 4437 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Bootstrap starting.
I20260806 04:37:49.442764 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.437638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44159 (local address 127.0.0.1:41964)
0806 04:37:49.437763 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.437778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.437858 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.438808 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.438811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.438832 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:49.438943 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.438949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.440404 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.440407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.441215 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.441224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.442252 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.442256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.442382 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.442455 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.442536 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.442629 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 04:37:49.442853 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.437808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.12.15.194:44159)
0806 04:37:49.438220 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.438233 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.438259 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:37:49.438342 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.438349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.438655 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.438824 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:49.439035 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.440294 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.441337 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.442098 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.442209 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.442360 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.442597 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.442692 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":191,"threads_started":1}
I20260806 04:37:49.443635 4437 tablet_bootstrap.cc:654] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:49.444315 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.437949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41069 (local address 127.0.0.1:46312)
0806 04:37:49.438200 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.438213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.438302 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.440138 (+ 1836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.440142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.440160 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:49.440248 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.440253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.441636 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.441640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.442717 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.442728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.443891 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.443895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.444007 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.444031 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.444116 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.444187 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260806 04:37:49.444851 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.438345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.12.15.195:41069)
0806 04:37:49.439496 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.439511 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.439536 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:37:49.439613 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.439618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.439810 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.439964 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:49.440409 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.441493 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.442867 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.443592 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.443691 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.443823 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.444613 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.444716 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":157,"threads_started":1}
I20260806 04:37:49.445343 4437 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: No bootstrap required, opened a new log
I20260806 04:37:49.445427 4437 ts_tablet_manager.cc:1397] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 04:37:49.445891 4437 raft_consensus.cc:348] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.446020 4437 raft_consensus.cc:374] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:49.446058 4437 raft_consensus.cc:729] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf49e05e753a4ad9b018ea85d591d229, State: Initialized, Role: FOLLOWER
I20260806 04:37:49.446185 4437 consensus_queue.cc:260] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.446450 4437 ts_tablet_manager.cc:1428] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 04:37:49.446316 4741 tablet_service.cc:1467] Processing CreateTablet for tablet 82c3916229f049f891d25a9dd4627b57 (DEFAULT_TABLE table=test-table [id=eb0a5a1a8c5e4288bf07d58a06f1bef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:49.447441 4741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c3916229f049f891d25a9dd4627b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:49.447651 4947 tablet_service.cc:1467] Processing CreateTablet for tablet 82c3916229f049f891d25a9dd4627b57 (DEFAULT_TABLE table=test-table [id=eb0a5a1a8c5e4288bf07d58a06f1bef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:37:49.448735 4947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c3916229f049f891d25a9dd4627b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:37:49.454452 5063 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Bootstrap starting.
I20260806 04:37:49.454897 5070 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Bootstrap starting.
I20260806 04:37:49.457026 5063 tablet_bootstrap.cc:654] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:49.457131 5070 tablet_bootstrap.cc:654] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Neither blocks nor log segments found. Creating new log.
I20260806 04:37:49.457852 5070 log.cc:824] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:49.457854 5063 log.cc:824] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Log is configured to *not* fsync() on all Append() calls
I20260806 04:37:49.459630 5070 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: No bootstrap required, opened a new log
I20260806 04:37:49.459633 5063 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: No bootstrap required, opened a new log
I20260806 04:37:49.459801 5063 ts_tablet_manager.cc:1397] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:37:49.459800 5070 ts_tablet_manager.cc:1397] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:37:49.462864 5070 raft_consensus.cc:348] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.462854 5063 raft_consensus.cc:348] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.463146 5063 raft_consensus.cc:374] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:49.463148 5070 raft_consensus.cc:374] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:37:49.463209 5063 raft_consensus.cc:729] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5708813ade1847c1a3285852a0059d3f, State: Initialized, Role: FOLLOWER
I20260806 04:37:49.463210 5070 raft_consensus.cc:729] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa917b8ccac4f5f832d0312ae717791, State: Initialized, Role: FOLLOWER
I20260806 04:37:49.463618 5070 consensus_queue.cc:260] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.463618 5063 consensus_queue.cc:260] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.464447 5070 ts_tablet_manager.cc:1428] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:37:49.464473 4845 heartbeater.cc:503] Master 127.12.15.254:43745 was elected leader, sending a full tablet report...
I20260806 04:37:49.464540 5063 ts_tablet_manager.cc:1428] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:37:49.464942 5044 heartbeater.cc:503] Master 127.12.15.254:43745 was elected leader, sending a full tablet report...
W20260806 04:37:49.503552 4623 tablet.cc:2369] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:37:49.635993 5045 tablet.cc:2369] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:37:49.697607 4847 tablet.cc:2369] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:37:49.794821 4631 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:37:49.795080 4631 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.796324 4631 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
I20260806 04:37:49.800312 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.796737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:44783 (local address 127.12.15.195:41069)
0806 04:37:49.796957 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.796962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.797076 (+ 114us) server_negotiation.cc:407] Connection header received
0806 04:37:49.797194 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.797198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.797243 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.797303 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:49.797661 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.798152 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.799329 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.799722 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.799831 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.799890 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.800113 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.800182 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 04:37:49.800319 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.796688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:40083 (local address 127.12.15.194:44159)
0806 04:37:49.796824 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.796829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.797026 (+ 197us) server_negotiation.cc:407] Connection header received
0806 04:37:49.797220 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.797223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.797268 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.797329 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:37:49.797661 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.798148 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.799332 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.799724 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.799831 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.799889 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.800114 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.800190 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:37:49.800383 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.796620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41069 (local address 127.12.15.193:44783)
0806 04:37:49.796979 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.796993 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.797103 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.797325 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.797328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.797344 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:49.797556 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.797562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.798256 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.798259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.799196 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.799207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.799832 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.799835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.799969 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.800017 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.800082 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.800175 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":188,"spinlock_wait_cycles":24960,"thread_start_us":77,"threads_started":1}
I20260806 04:37:49.800424 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.796616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44159 (local address 127.12.15.193:40083)
0806 04:37:49.796924 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.796938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:37:49.797071 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:37:49.797355 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:37:49.797358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:37:49.797402 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:37:49.797543 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.797549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.798253 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.798257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.799198 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:37:49.799209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.799833 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.799837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:37:49.799967 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.800014 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 04:37:49.800080 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 04:37:49.800190 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":137,"spinlock_wait_cycles":2176,"thread_start_us":100,"threads_started":1}
I20260806 04:37:49.801139 4769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3916229f049f891d25a9dd4627b57" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5708813ade1847c1a3285852a0059d3f" is_pre_election: true
I20260806 04:37:49.801141 4982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3916229f049f891d25a9dd4627b57" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa917b8ccac4f5f832d0312ae717791" is_pre_election: true
I20260806 04:37:49.801481 4769 raft_consensus.cc:2454] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 0.
I20260806 04:37:49.801481 4982 raft_consensus.cc:2454] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 0.
I20260806 04:37:49.802014 4429 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa917b8ccac4f5f832d0312ae717791, bf49e05e753a4ad9b018ea85d591d229; no voters:
I20260806 04:37:49.802325 4631 raft_consensus.cc:2781] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:37:49.802416 4631 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:37:49.802449 4631 raft_consensus.cc:3037] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:49.803473 4631 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.803820 4631 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 election: Requested vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
I20260806 04:37:49.804231 4769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3916229f049f891d25a9dd4627b57" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5708813ade1847c1a3285852a0059d3f"
I20260806 04:37:49.804231 4982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3916229f049f891d25a9dd4627b57" candidate_uuid: "bf49e05e753a4ad9b018ea85d591d229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa917b8ccac4f5f832d0312ae717791"
I20260806 04:37:49.804400 4769 raft_consensus.cc:3037] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:49.804397 4982 raft_consensus.cc:3037] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:37:49.805721 4769 raft_consensus.cc:2454] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 1.
I20260806 04:37:49.805721 4982 raft_consensus.cc:2454] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf49e05e753a4ad9b018ea85d591d229 in term 1.
I20260806 04:37:49.806048 4429 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa917b8ccac4f5f832d0312ae717791, bf49e05e753a4ad9b018ea85d591d229; no voters:
I20260806 04:37:49.806221 4631 raft_consensus.cc:2781] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:37:49.806403 4631 raft_consensus.cc:686] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 LEADER]: Becoming Leader. State: Replica: bf49e05e753a4ad9b018ea85d591d229, State: Running, Role: LEADER
I20260806 04:37:49.806699 4631 consensus_queue.cc:237] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:37:49.809638 4318 catalog_manager.cc:5495] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "bf49e05e753a4ad9b018ea85d591d229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } health_report { overall_health: HEALTHY } } }
I20260806 04:37:49.898321 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.894856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.12.15.193:38231)
0806 04:37:49.895163 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 04:37:49.895169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:37:49.895181 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:37:49.895259 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:37:49.895261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:37:49.895316 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:37:49.895412 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:37:49.895662 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.896167 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.896970 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:37:49.897206 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:37:49.897298 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:37:49.897483 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:37:49.897968 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:37:49.898069 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:37:49.898140 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 04:37:49.898195 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":138,"threads_started":1}
I20260806 04:37:49.902503 4769 raft_consensus.cc:1261] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Refusing update from remote peer bf49e05e753a4ad9b018ea85d591d229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:49.902503 4982 raft_consensus.cc:1261] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Refusing update from remote peer bf49e05e753a4ad9b018ea85d591d229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:37:49.903229 4631 consensus_queue.cc:1035] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:49.903429 5079 consensus_queue.cc:1035] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:37:49.912770 5090 mvcc.cc:205] Tried to move back new op lower bound from 7315419422314270720 to 7315419421930557440. Current Snapshot: MvccSnapshot[applied={T|T < 7315419422314270720}]
I20260806 04:37:49.919399 4319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:49.919567 4319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:37:49.921988 4319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Sending DeleteTablet for 3 replicas of tablet 82c3916229f049f891d25a9dd4627b57
I20260806 04:37:49.922868 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 4375
I20260806 04:37:49.922819 4947 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c3916229f049f891d25a9dd4627b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:49 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:46312
I20260806 04:37:49.923099 4741 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c3916229f049f891d25a9dd4627b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:37:49 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:41964
I20260806 04:37:49.928686 5106 tablet_replica.cc:330] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: stopping tablet replica
I20260806 04:37:49.929111 5106 raft_consensus.cc:2229] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:49.929517 5106 raft_consensus.cc:2258] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:49.929939 5107 tablet_replica.cc:330] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: stopping tablet replica
I20260806 04:37:49.930415 5107 raft_consensus.cc:2229] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:37:49.930835 5107 raft_consensus.cc:2258] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:37:49.931882 5106 ts_tablet_manager.cc:1905] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:49.932080 5107 ts_tablet_manager.cc:1905] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:37:49.934243 5107 ts_tablet_manager.cc:1918] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:49.934336 5107 log.cc:1196] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/82c3916229f049f891d25a9dd4627b57
I20260806 04:37:49.934626 5107 ts_tablet_manager.cc:1939] T 82c3916229f049f891d25a9dd4627b57 P 5708813ade1847c1a3285852a0059d3f: Deleting consensus metadata
I20260806 04:37:49.934844 5106 ts_tablet_manager.cc:1918] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:37:49.935235 5106 log.cc:1196] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/82c3916229f049f891d25a9dd4627b57
I20260806 04:37:49.935595 5106 ts_tablet_manager.cc:1939] T 82c3916229f049f891d25a9dd4627b57 P 5fa917b8ccac4f5f832d0312ae717791: Deleting consensus metadata
I20260806 04:37:49.935899 4296 catalog_manager.cc:4841] TS 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): tablet 82c3916229f049f891d25a9dd4627b57 (table test-table [id=eb0a5a1a8c5e4288bf07d58a06f1bef5]) successfully deleted
I20260806 04:37:49.936746 4295 catalog_manager.cc:4841] TS 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): tablet 82c3916229f049f891d25a9dd4627b57 (table test-table [id=eb0a5a1a8c5e4288bf07d58a06f1bef5]) successfully deleted
I20260806 04:37:49.938468 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 4628
W20260806 04:37:49.938452 4296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:38231 (error 108)
I20260806 04:37:49.945545 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:37:49.945161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38231 (local address 127.0.0.1:54382)
0806 04:37:49.945380 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 04:37:49.945448 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":60,"threads_started":1}
W20260806 04:37:49.945715 4296 catalog_manager.cc:4584] TS bf49e05e753a4ad9b018ea85d591d229 (127.12.15.193:38231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82c3916229f049f891d25a9dd4627b57: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38231: connect: Connection refused (error 111)
I20260806 04:37:49.950161 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 4851
I20260806 04:37:49.961683 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 4271
I20260806 04:38:09.977484 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43745
--webserver_interface=127.12.15.254
--webserver_port=37593
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:43745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:10.154615 5111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:10.155028 5111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:10.155095 5111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:10.161324 5111 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 04:38:10.161389 5111 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 04:38:10.161410 5111 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 04:38:10.161427 5111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:10.161451 5111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:10.161482 5111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:10.161514 5111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:10.161550 5111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:10.169639 5111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:43745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=37593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:10.171950 5111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:10.173368 5111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:10.187489 5120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:10.187503 5125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:10.187510 5121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:10.188186 5111 server_base.cc:1034] running on GCE node
I20260806 04:38:10.189143 5111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:10.191576 5111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:10.192903 5111 hybrid_clock.cc:648] HybridClock initialized: now 1785991090192856 us; error 36 us; skew 500 ppm
Aug 06 04:38:10 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991090, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:10.197569 5111 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:38:10.199712 5111 webserver.cc:460] Webserver started at http://127.12.15.254:37593/ using document root <none> and password file <none>
I20260806 04:38:10.201210 5111 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:10.201284 5111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:10.207134 5111 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 04:38:10.209381 5133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.210649 5111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:10.210772 5111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "b2888eb3517247efb42e46ba8c00c9d2"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "4c6e8ef24de5b6fb5911fb11a168945d"
server_key_iv: "4a2a9ededa2e11fc9cdc55de3cf3080d"
server_key_version: "encryptionkey@0"
I20260806 04:38:10.211901 5111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:10.239720 5111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:10.242833 5111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:10.242987 5111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:10.249897 5111 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:43745
I20260806 04:38:10.250154 5216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:43745 every 8 connection(s)
I20260806 04:38:10.250929 5111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:10.255296 5217 sys_catalog.cc:263] Verifying existing consensus state
I20260806 04:38:10.256999 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap starting.
I20260806 04:38:10.259008 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 5111
I20260806 04:38:10.259524 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:38231
--local_ip_for_outbound_sockets=127.12.15.193
--tserver_master_addrs=127.12.15.254:43745
--webserver_port=38581
--webserver_interface=127.12.15.193
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:38:10.275005 5217 log.cc:824] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:10.286258 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:38:10.286631 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Bootstrap complete.
I20260806 04:38:10.290539 5217 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:38:10.290925 5217 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2888eb3517247efb42e46ba8c00c9d2, State: Initialized, Role: FOLLOWER
I20260806 04:38:10.291396 5217 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:38:10.291512 5217 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:10.291574 5217 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:10.291677 5217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 2 FOLLOWER]: Advancing to term 3
I20260806 04:38:10.293565 5217 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:38:10.293889 5217 leader_election.cc:302] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2888eb3517247efb42e46ba8c00c9d2; no voters:
I20260806 04:38:10.294183 5217 leader_election.cc:288] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 04:38:10.294306 5225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 04:38:10.294622 5225 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [term 3 LEADER]: Becoming Leader. State: Replica: b2888eb3517247efb42e46ba8c00c9d2, State: Running, Role: LEADER
I20260806 04:38:10.294931 5225 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } }
I20260806 04:38:10.295447 5217 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:10.295885 5225 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2888eb3517247efb42e46ba8c00c9d2. Latest consensus state: current_term: 3 leader_uuid: "b2888eb3517247efb42e46ba8c00c9d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } } }
I20260806 04:38:10.295996 5225 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:10.296399 5231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:10.296482 5228 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b2888eb3517247efb42e46ba8c00c9d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2888eb3517247efb42e46ba8c00c9d2" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 43745 } } }
I20260806 04:38:10.296674 5228 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:10.301570 5231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:10.302232 5231 catalog_manager.cc:1259] Loaded cluster ID: b8c14d6fc56149daa0fb186720ce8e5d
I20260806 04:38:10.302433 5231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:10.305012 5231 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:10.305817 5231 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Loaded TSK: 0
I20260806 04:38:10.305902 5231 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2888eb3517247efb42e46ba8c00c9d2: Loaded TSK: 1
F20260806 04:38:10.306627 5231 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f842fb709ad google::LogMessage::Fail() at ??:0
@ 0x7f842fb7388a google::LogMessage::SendToLog() at ??:0
@ 0x7f842fb704de google::LogMessage::Flush() at ??:0
@ 0x7f842fb71a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f843283418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f843282cf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f843a4d57bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f843a4af99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f843a4b0178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f843a4f1df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f843a51082b std::function<>::operator()() at ??:0
@ 0x7f843a4afb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f843a4b0f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f843a4ad579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f843a4f1640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8431dc23d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8431dc2c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8431dc43f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8431db3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f84333f76db start_thread at ??:0
@ 0x7f842e66e61f clone at ??:0
W20260806 04:38:10.399824 5223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:10.400218 5223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:10.400292 5223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:10.406414 5223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:10.406500 5223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:10.406592 5223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:10.414615 5223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:38231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=38581
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:10.416065 5223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:10.417502 5223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:10.428819 5293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:10.428843 5292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:10.428921 5223 server_base.cc:1034] running on GCE node
W20260806 04:38:10.429246 5295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:10.429621 5223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:10.430245 5223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:10.431427 5223 hybrid_clock.cc:648] HybridClock initialized: now 1785991090431390 us; error 53 us; skew 500 ppm
Aug 06 04:38:10 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991090, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:10.434063 5223 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:38:10.435007 5223 webserver.cc:460] Webserver started at http://127.12.15.193:38581/ using document root <none> and password file <none>
I20260806 04:38:10.435571 5223 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:10.435633 5223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:10.439782 5223 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:38:10.442489 5305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.443812 5223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:38:10.443940 5223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "bf49e05e753a4ad9b018ea85d591d229"
format_stamp: "Formatted at 2026-08-06 04:37:27 on dist-test-slave-xt4k"
server_key: "14ef70885810554ae4e673a6aba6a456"
server_key_iv: "a81e794405b5a760e2932130fc5fe9d3"
server_key_version: "encryptionkey@0"
I20260806 04:38:10.444522 5223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:10.456825 5223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:10.459887 5223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:10.460064 5223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:10.460729 5223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:10.461872 5318 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 04:38:10.466032 5223 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 04:38:10.466087 5223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 04:38:10.466147 5223 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 04:38:10.467878 5223 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 04:38:10.467936 5223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 04:38:10.468227 5318 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Bootstrap starting.
I20260806 04:38:10.478243 5223 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38231
I20260806 04:38:10.478549 5481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38231 every 8 connection(s)
I20260806 04:38:10.479478 5223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:10.487581 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 5223
I20260806 04:38:10.488157 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:44159
--local_ip_for_outbound_sockets=127.12.15.194
--tserver_master_addrs=127.12.15.254:43745
--webserver_port=35547
--webserver_interface=127.12.15.194
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:38:10.488476 5318 log.cc:824] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:10.497637 5318 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 04:38:10.498070 5318 tablet_bootstrap.cc:492] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Bootstrap complete.
I20260806 04:38:10.499415 5318 ts_tablet_manager.cc:1397] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.008s
I20260806 04:38:10.502851 5318 raft_consensus.cc:348] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:10.503393 5318 raft_consensus.cc:729] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf49e05e753a4ad9b018ea85d591d229, State: Initialized, Role: FOLLOWER
I20260806 04:38:10.503921 5318 consensus_queue.cc:260] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:10.504978 5318 ts_tablet_manager.cc:1428] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260806 04:38:10.540123 5488 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:38:10.481920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:33975)
0806 04:38:10.482539 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.482600 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:10.483663 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:10.538993 (+ 55330us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":162,"threads_started":1}
W20260806 04:38:10.540783 5482 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:43745 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.254:43745: Client connection negotiation failed: client connection to 127.12.15.254:43745: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 04:38:10.541546 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.541268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:46693)
0806 04:38:10.541400 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.541471 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 04:38:10.542181 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.541974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:45887)
0806 04:38:10.542075 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.542124 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 04:38:10.542405 5482 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:38:10.611248 5492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:10.611505 5492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:10.611572 5492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:10.615497 5492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:10.615576 5492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:10.615656 5492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:10.620777 5492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:44159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=35547
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:10.621767 5492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:10.622766 5492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:10.632158 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:10.632211 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:10.632192 5492 server_base.cc:1034] running on GCE node
W20260806 04:38:10.632298 5504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:10.632833 5492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:10.633453 5492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:10.634608 5492 hybrid_clock.cc:648] HybridClock initialized: now 1785991090634589 us; error 32 us; skew 500 ppm
Aug 06 04:38:10 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991090, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:10.637264 5492 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:38:10.638156 5492 webserver.cc:460] Webserver started at http://127.12.15.194:35547/ using document root <none> and password file <none>
I20260806 04:38:10.638748 5492 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:10.638818 5492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:10.642628 5492 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 04:38:10.644661 5514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.645849 5492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:38:10.645958 5492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "5708813ade1847c1a3285852a0059d3f"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "41b01c465aa8653df0eb0de9ff34c2e8"
server_key_iv: "843a808a9799b68643cae0a27bd0a69e"
server_key_version: "encryptionkey@0"
I20260806 04:38:10.646483 5492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:10.679734 5492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:10.683219 5492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:10.683442 5492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:10.684087 5492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:10.685014 5492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:10.685065 5492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.685119 5492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:10.685149 5492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.694617 5492 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:44159
I20260806 04:38:10.694664 5686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:44159 every 8 connection(s)
I20260806 04:38:10.695708 5492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:10.697844 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 5492
I20260806 04:38:10.698268 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:41069
--local_ip_for_outbound_sockets=127.12.15.195
--tserver_master_addrs=127.12.15.254:43745
--webserver_port=46523
--webserver_interface=127.12.15.195
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:38:10.699831 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.697938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:49629)
0806 04:38:10.698463 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.698598 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":109,"threads_started":1}
W20260806 04:38:10.700307 5687 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:43745 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.254:43745: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
I20260806 04:38:10.701028 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.700784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:33791)
0806 04:38:10.700909 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.700970 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 04:38:10.701689 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.701475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:35249)
0806 04:38:10.701593 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.701636 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 04:38:10.701856 5687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 04:38:10.812721 5690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:10.812969 5690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:10.813010 5690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:10.817013 5690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:10.817065 5690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:10.817119 5690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:10.822157 5690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:41069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=46523
--tserver_master_addrs=127.12.15.254:43745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:10.823137 5690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:10.824097 5690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:10.833461 5706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:10.833444 5702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:10.833489 5701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:10.833726 5690 server_base.cc:1034] running on GCE node
I20260806 04:38:10.834123 5690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:10.834743 5690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:10.835892 5690 hybrid_clock.cc:648] HybridClock initialized: now 1785991090835865 us; error 42 us; skew 500 ppm
Aug 06 04:38:10 dist-test-slave-xt4k krb5kdc[3397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991090, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:10.838670 5690 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:38:10.839519 5690 webserver.cc:460] Webserver started at http://127.12.15.195:46523/ using document root <none> and password file <none>
I20260806 04:38:10.840061 5690 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:10.840128 5690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:10.843925 5690 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:10.846089 5718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.847332 5690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:10.847445 5690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "5fa917b8ccac4f5f832d0312ae717791"
format_stamp: "Formatted at 2026-08-06 04:37:28 on dist-test-slave-xt4k"
server_key: "2a525107082881d604efb1dec3961732"
server_key_iv: "d974b0168c4c73ace191c658ef98df0e"
server_key_version: "encryptionkey@0"
I20260806 04:38:10.848024 5690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:10.861066 5690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:10.864112 5690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:10.864295 5690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:10.864934 5690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:10.865804 5690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:10.865854 5690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.865909 5690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:10.865937 5690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:10.875125 5690 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:41069
I20260806 04:38:10.875160 5893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:41069 every 8 connection(s)
I20260806 04:38:10.876173 5690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:38:10.880652 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.878404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:44435)
0806 04:38:10.879074 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.879235 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":123,"threads_started":1}
W20260806 04:38:10.881089 5896 heartbeater.cc:644] Failed to heartbeat to 127.12.15.254:43745 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.254:43745: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
I20260806 04:38:10.881668 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.881454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:45905)
0806 04:38:10.881561 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.881613 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 04:38:10.882289 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:10.882096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:50263)
0806 04:38:10.882194 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:10.882238 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 04:38:10.882508 5896 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 04:38:10.886209 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 5690
I20260806 04:38:11.543457 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:11.542990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:46219)
0806 04:38:11.543280 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:11.543337 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260806 04:38:11.702824 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:11.702383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:45825)
0806 04:38:11.702656 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:11.702719 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260806 04:38:11.883488 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:11.883046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:55303)
0806 04:38:11.883299 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:11.883366 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260806 04:38:12.041471 5904 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:12.041680 5904 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:12.042817 5904 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
Aug 06 04:38:12 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991090, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.194@KRBTEST.COM
Aug 06 04:38:12 dist-test-slave-xt4k krb5kdc[3397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991090, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.195@KRBTEST.COM
I20260806 04:38:12.056200 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.043011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44159 (local address 127.12.15.193:34719)
0806 04:38:12.043194 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:12.043209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:12.043315 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:12.045033 (+ 1718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:12.045038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:12.045572 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:12.046046 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:12.046065 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.047280 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.047283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:12.048305 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:12.048314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.049097 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.049100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:12.049278 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:12.049862 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:12.049884 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:12.051678 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:12.053682 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:12.053688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:12.053701 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:12.054036 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:12.054334 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:12.054339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:12.054498 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:12.055222 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:12.055228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:12.055447 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 04:38:12.055664 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 04:38:12.056069 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":250,"spinlock_wait_cycles":23040}
I20260806 04:38:12.056247 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.043246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:53827 (local address 127.12.15.195:41069)
0806 04:38:12.043570 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:12.043595 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:12.043634 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:38:12.044306 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:12.044312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:12.044691 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:12.044964 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:12.046198 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.047147 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.048422 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.048980 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.049126 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:12.052201 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:12.052208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:12.052222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:12.052254 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:12.054093 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054508 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:12.054513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:12.054517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:12.054651 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054889 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:12.054892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:12.054894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:12.055353 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:12.055449 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:12.055787 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 04:38:12.056080 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260806 04:38:12.056247 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.043277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:34719 (local address 127.12.15.194:44159)
0806 04:38:12.043694 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:12.043714 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:12.043746 (+ 32us) server_negotiation.cc:407] Connection header received
0806 04:38:12.044399 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:12.044405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:12.044692 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:12.044973 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:12.046208 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.047159 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.048472 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.048978 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.049128 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:12.051891 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:12.051895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:12.051907 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:12.051932 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:12.053563 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054138 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:12.054142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:12.054145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:12.054250 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054598 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:12.054601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:12.054603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:12.055088 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:12.055221 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:12.055787 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 04:38:12.056066 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":93,"threads_started":1}
I20260806 04:38:12.056198 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.043070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:41069 (local address 127.12.15.193:53827)
0806 04:38:12.043383 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:12.043397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:12.043506 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:12.045032 (+ 1526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:12.045038 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:12.045569 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:12.046046 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:12.046060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.047279 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.047282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:12.048302 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:12.048312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:12.049315 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:12.049318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:12.049404 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:12.049986 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:12.049998 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:12.051993 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:12.054203 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:12.054207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:12.054411 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:12.054743 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:12.054745 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:12.054746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:12.054815 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:12.055494 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:12.055497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:12.055574 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 04:38:12.055664 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 04:38:12.056003 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":177,"thread_start_us":75,"threads_started":1}
W20260806 04:38:12.058759 5307 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
W20260806 04:38:12.058892 5309 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
I20260806 04:38:12.058964 5309 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf49e05e753a4ad9b018ea85d591d229; no voters: 5708813ade1847c1a3285852a0059d3f, 5fa917b8ccac4f5f832d0312ae717791
I20260806 04:38:12.059175 5904 raft_consensus.cc:2726] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:38:12.544582 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.544229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:45337)
0806 04:38:12.544425 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:12.544492 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 04:38:12.703922 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.703555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:33407)
0806 04:38:12.703779 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:12.703831 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260806 04:38:12.885026 5910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:12.884322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:37547)
0806 04:38:12.884839 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:12.884905 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":95,"threads_started":1}
I20260806 04:38:13.545855 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:13.545403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:48703)
0806 04:38:13.545694 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:13.545757 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260806 04:38:13.652036 5912 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:13.652177 5912 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:13.652607 5912 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
W20260806 04:38:13.653352 5309 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
W20260806 04:38:13.653457 5307 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
I20260806 04:38:13.653546 5307 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf49e05e753a4ad9b018ea85d591d229; no voters: 5708813ade1847c1a3285852a0059d3f, 5fa917b8ccac4f5f832d0312ae717791
I20260806 04:38:13.653700 5912 raft_consensus.cc:2726] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:38:13.705071 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:13.704628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:41807)
0806 04:38:13.704910 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:13.704970 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260806 04:38:13.886371 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:13.885872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:55371)
0806 04:38:13.886196 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:13.886265 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":82,"threads_started":1}
I20260806 04:38:14.547067 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:14.546623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:35953)
0806 04:38:14.546898 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:14.546967 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260806 04:38:14.706202 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:14.705793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:54085)
0806 04:38:14.706049 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:14.706107 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260806 04:38:14.887568 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:14.887153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:56461)
0806 04:38:14.887412 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:14.887473 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":80,"threads_started":1}
I20260806 04:38:15.548454 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:15.547969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:44313)
0806 04:38:15.548262 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:15.548319 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":75,"threads_started":1}
I20260806 04:38:15.707455 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:15.706961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:56567)
0806 04:38:15.707300 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:15.707360 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":134,"threads_started":1}
I20260806 04:38:15.888801 5921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:15.888381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:60847)
0806 04:38:15.888640 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:15.888699 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260806 04:38:16.521700 5922 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:16.521845 5922 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:16.522318 5922 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
W20260806 04:38:16.523192 5309 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
W20260806 04:38:16.523315 5307 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
I20260806 04:38:16.523379 5307 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf49e05e753a4ad9b018ea85d591d229; no voters: 5708813ade1847c1a3285852a0059d3f, 5fa917b8ccac4f5f832d0312ae717791
I20260806 04:38:16.523530 5922 raft_consensus.cc:2726] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:38:16.549784 5924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:16.549193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:35737)
0806 04:38:16.549571 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:16.549645 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":100,"threads_started":1}
I20260806 04:38:16.708806 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:16.708311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:52931)
0806 04:38:16.708629 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:16.708690 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":79,"threads_started":1}
I20260806 04:38:16.890008 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:16.889528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:34557)
0806 04:38:16.889850 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:16.889910 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":96,"threads_started":1}
I20260806 04:38:17.551116 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:17.550609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:39409)
0806 04:38:17.550938 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:17.551010 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
I20260806 04:38:17.710148 5933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:17.709580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:49615)
0806 04:38:17.709975 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:17.710039 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":150,"threads_started":1}
I20260806 04:38:17.891321 5934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:17.890807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:57175)
0806 04:38:17.891156 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:17.891220 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":93,"threads_started":1}
I20260806 04:38:18.552390 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:18.551937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:47315)
0806 04:38:18.552189 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:18.552250 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
I20260806 04:38:18.711292 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:18.710867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:49221)
0806 04:38:18.711143 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:18.711197 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260806 04:38:18.892779 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:18.892240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:60905)
0806 04:38:18.892592 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:18.892659 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260806 04:38:19.553459 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:19.553071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:39515)
0806 04:38:19.553309 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:19.553362 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
I20260806 04:38:19.712489 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:19.712017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:43905)
0806 04:38:19.712287 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:19.712369 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260806 04:38:19.893998 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:19.893548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:35381)
0806 04:38:19.893827 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:19.893887 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":83,"threads_started":1}
I20260806 04:38:20.554919 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:20.554461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:56107)
0806 04:38:20.554749 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:20.554815 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260806 04:38:20.713898 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:20.713428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:38957)
0806 04:38:20.713730 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:20.713797 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":87,"threads_started":1}
I20260806 04:38:20.895385 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:20.894891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:60411)
0806 04:38:20.895212 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:20.895281 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260806 04:38:21.310814 5952 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:21.310976 5952 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:21.311422 5952 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
W20260806 04:38:21.312287 5309 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
W20260806 04:38:21.312425 5307 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
I20260806 04:38:21.312513 5307 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf49e05e753a4ad9b018ea85d591d229; no voters: 5708813ade1847c1a3285852a0059d3f, 5fa917b8ccac4f5f832d0312ae717791
I20260806 04:38:21.312676 5952 raft_consensus.cc:2726] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:38:21.556214 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:21.555723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:46751)
0806 04:38:21.556038 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:21.556110 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":92,"threads_started":1}
I20260806 04:38:21.715175 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:21.714727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:55045)
0806 04:38:21.715011 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:21.715073 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
I20260806 04:38:21.896593 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:21.896170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:36727)
0806 04:38:21.896432 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:21.896492 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260806 04:38:22.557484 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:22.557014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:60405)
0806 04:38:22.557299 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:22.557362 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260806 04:38:22.716464 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:22.715984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:49205)
0806 04:38:22.716260 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:22.716323 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260806 04:38:22.897794 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:22.897341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:58241)
0806 04:38:22.897622 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:22.897682 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260806 04:38:23.558646 5964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:23.558232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:50777)
0806 04:38:23.558487 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:23.558546 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260806 04:38:23.717571 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:23.717155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:49975)
0806 04:38:23.717416 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:23.717472 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260806 04:38:23.898996 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:23.898577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:39247)
0806 04:38:23.898840 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:23.898898 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260806 04:38:24.559849 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:24.559379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:49265)
0806 04:38:24.559674 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:24.559743 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
I20260806 04:38:24.718750 5970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:24.718281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:56799)
0806 04:38:24.718579 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:24.718640 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
I20260806 04:38:24.900141 5971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:24.899713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:39507)
0806 04:38:24.899966 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:24.900031 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260806 04:38:25.460875 5972 raft_consensus.cc:482] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:25.461009 5972 raft_consensus.cc:504] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5708813ade1847c1a3285852a0059d3f" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44159 } } peers { permanent_uuid: "5fa917b8ccac4f5f832d0312ae717791" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 41069 } } peers { permanent_uuid: "bf49e05e753a4ad9b018ea85d591d229" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38231 } }
I20260806 04:38:25.461424 5972 leader_election.cc:288] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159), 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069)
W20260806 04:38:25.462136 5307 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fa917b8ccac4f5f832d0312ae717791 (127.12.15.195:41069): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
W20260806 04:38:25.462246 5309 leader_election.cc:341] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5708813ade1847c1a3285852a0059d3f (127.12.15.194:44159): Not found: Tablet not found: 82c3916229f049f891d25a9dd4627b57
I20260806 04:38:25.462311 5309 leader_election.cc:302] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf49e05e753a4ad9b018ea85d591d229; no voters: 5708813ade1847c1a3285852a0059d3f, 5fa917b8ccac4f5f832d0312ae717791
I20260806 04:38:25.462447 5972 raft_consensus.cc:2726] T 82c3916229f049f891d25a9dd4627b57 P bf49e05e753a4ad9b018ea85d591d229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 04:38:25.561319 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:25.560838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.193:54347)
0806 04:38:25.561136 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:25.561204 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260806 04:38:25.720065 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:25.719642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.194:60657)
0806 04:38:25.719911 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:25.719967 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":92,"threads_started":1}
I20260806 04:38:25.887400 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 5223
I20260806 04:38:25.901492 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:25.901065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:43745 (local address 127.12.15.195:56451)
0806 04:38:25.901321 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:25.901380 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.254:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260806 04:38:25.907809 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 5492
I20260806 04:38:25.929131 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 5690
2026-08-06T04:38:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59804 ms)
[----------] 30 tests from SecurityITest (246887 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:26 dist-test-slave-xt4k krb5kdc[5985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:26 dist-test-slave-xt4k krb5kdc[5985](info): set up 2 sockets
Aug 06 04:38:26 dist-test-slave-xt4k krb5kdc[5985](info): commencing operation
krb5kdc: starting...
W20260806 04:38:28.040692 12351 mini_kdc.cc:121] Time spent starting KDC: real 2.084s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:38:28.169552 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:41007
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:41007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:28.284749 6007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:28.285048 6007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:28.285105 6007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:28.289137 6007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:28.289193 6007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:28.289219 6007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:28.289242 6007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:28.289268 6007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:28.294844 6007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:41007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:41007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:28.295929 6007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:28.297008 6007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:28.305846 6020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:28.305848 6022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:28.305856 6019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:28.305907 6007 server_base.cc:1034] running on GCE node
I20260806 04:38:28.306850 6007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:28.308066 6007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:28.309248 6007 hybrid_clock.cc:648] HybridClock initialized: now 1785991108309222 us; error 46 us; skew 500 ppm
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:28.312211 6007 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:38:28.313230 6007 webserver.cc:460] Webserver started at http://127.12.15.254:33407/ using document root <none> and password file <none>
I20260806 04:38:28.313776 6007 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:28.313834 6007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:28.314019 6007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:28.316094 6007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "371615d0519243bf9c56b1fb2391b7a9"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "b42dbd44f689ce1c49fcf6c94ea94bfc"
server_key_iv: "49acb57034d4cccd435505f95ee1c48f"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.316659 6007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "371615d0519243bf9c56b1fb2391b7a9"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "b42dbd44f689ce1c49fcf6c94ea94bfc"
server_key_iv: "49acb57034d4cccd435505f95ee1c48f"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.320394 6007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:38:28.322918 6033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.324224 6007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:28.324359 6007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "371615d0519243bf9c56b1fb2391b7a9"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "b42dbd44f689ce1c49fcf6c94ea94bfc"
server_key_iv: "49acb57034d4cccd435505f95ee1c48f"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.324470 6007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:28.338471 6007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:28.341450 6007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:28.341629 6007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:28.348840 6007 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:41007
I20260806 04:38:28.348867 6117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:41007 every 8 connection(s)
I20260806 04:38:28.349869 6007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:28.352798 6118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:28.357084 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 6007
I20260806 04:38:28.357214 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:28.357472 12351 external_mini_cluster.cc:1428] Setting key 9e07976edca3e43663d6dce3648361d6
I20260806 04:38:28.358948 6118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9: Bootstrap starting.
I20260806 04:38:28.361716 6118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:28.362527 6118 log.cc:824] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:28.364432 6118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9: No bootstrap required, opened a new log
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:28.368428 6118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371615d0519243bf9c56b1fb2391b7a9" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41007 } }
I20260806 04:38:28.368671 6118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:28.368723 6118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371615d0519243bf9c56b1fb2391b7a9, State: Initialized, Role: FOLLOWER
I20260806 04:38:28.369156 6118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371615d0519243bf9c56b1fb2391b7a9" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41007 } }
I20260806 04:38:28.369287 6118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:28.369352 6118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:28.369450 6118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:28.370328 6118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371615d0519243bf9c56b1fb2391b7a9" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41007 } }
I20260806 04:38:28.370707 6118 leader_election.cc:302] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 371615d0519243bf9c56b1fb2391b7a9; no voters:
I20260806 04:38:28.371079 6118 leader_election.cc:288] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:28.371177 6126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:28.371440 6126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [term 1 LEADER]: Becoming Leader. State: Replica: 371615d0519243bf9c56b1fb2391b7a9, State: Running, Role: LEADER
I20260806 04:38:28.371822 6126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371615d0519243bf9c56b1fb2391b7a9" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41007 } }
I20260806 04:38:28.372248 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:28.358899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.12.15.254:41007)
0806 04:38:28.359357 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:28.359385 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:28.359427 (+ 42us) server_negotiation.cc:407] Connection header received
0806 04:38:28.360280 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:28.360306 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:28.360753 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:28.361133 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:28.361867 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.363056 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.363914 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.364438 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.364584 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:28.367252 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:28.367258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:28.367273 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:28.367300 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:28.369132 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:28.369632 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:28.369639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:28.369645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:28.369764 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:28.370040 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:28.370043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:28.370045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:28.370454 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:28.370633 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:28.370948 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 04:38:28.371257 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":135,"threads_started":1}
I20260806 04:38:28.372391 6118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:28.373154 6126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 371615d0519243bf9c56b1fb2391b7a9. Latest consensus state: current_term: 1 leader_uuid: "371615d0519243bf9c56b1fb2391b7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371615d0519243bf9c56b1fb2391b7a9" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41007 } } }
I20260806 04:38:28.373353 6126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:28.375037 6142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:28.375319 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "371615d0519243bf9c56b1fb2391b7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "371615d0519243bf9c56b1fb2391b7a9" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 41007 } } }
I20260806 04:38:28.375497 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:28.378300 6142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:28.385825 6142 catalog_manager.cc:1347] Generated new cluster ID: 6b7cdaeac8804d0d851c87e63264fe84
I20260806 04:38:28.385917 6142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:28.403751 6142 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:28.404834 6142 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:28.409107 6142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9: Generated new TSK 0
I20260806 04:38:28.409808 6142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:38:28.444002 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41007
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:28.589725 6167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:28.589969 6167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:28.590010 6167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:28.593915 6167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:28.593971 6167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:28.594025 6167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:28.599102 6167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:28.600021 6167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:28.600988 6167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:28.610363 6179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:28.610385 6175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:28.610345 6181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:28.610530 6167 server_base.cc:1034] running on GCE node
I20260806 04:38:28.610993 6167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:28.611613 6167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:28.612764 6167 hybrid_clock.cc:648] HybridClock initialized: now 1785991108612745 us; error 30 us; skew 500 ppm
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:28.615388 6167 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:38:28.616242 6167 webserver.cc:460] Webserver started at http://127.12.15.193:32841/ using document root <none> and password file <none>
I20260806 04:38:28.616811 6167 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:28.616878 6167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:28.617069 6167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:28.619071 6167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "57bb2b06aa694b32a43f6c6976cf5ef4"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "41a3fe97cb3a955b8e9a303584fdc74e"
server_key_iv: "bd488f78fc4c0e566a0bf35810ec5fbd"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.619614 6167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "57bb2b06aa694b32a43f6c6976cf5ef4"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "41a3fe97cb3a955b8e9a303584fdc74e"
server_key_iv: "bd488f78fc4c0e566a0bf35810ec5fbd"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.623311 6167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:38:28.625682 6193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.626883 6167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:28.627007 6167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "57bb2b06aa694b32a43f6c6976cf5ef4"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "41a3fe97cb3a955b8e9a303584fdc74e"
server_key_iv: "bd488f78fc4c0e566a0bf35810ec5fbd"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.627102 6167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:28.646159 6167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:28.649109 6167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:28.649294 6167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:28.649909 6167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:28.650756 6167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:28.650804 6167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.650859 6167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:28.650884 6167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.660086 6167 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:46425
I20260806 04:38:28.660115 6380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:46425 every 8 connection(s)
I20260806 04:38:28.661048 6167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:28.662251 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 6167
I20260806 04:38:28.662375 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:38:28.662714 12351 external_mini_cluster.cc:1428] Setting key 6b89d4bde110bf71a4b01a1faed7ed64
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:38:28.676586 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:28.664379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:57015 (local address 127.12.15.254:41007)
0806 04:38:28.664505 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:28.664510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:28.664527 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:38:28.665583 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:28.665586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:28.665650 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:28.665723 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:28.667225 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.667764 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.668979 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.669213 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.669308 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:28.672061 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:28.672066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:28.672069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:28.672101 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:28.673944 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:28.674608 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:28.674612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:28.674613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:28.674698 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:28.675135 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:28.675139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:28.675140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:28.675377 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:28.675485 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:28.676274 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 04:38:28.676441 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:38:28.677757 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:28.663472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41007 (local address 127.12.15.193:57015)
0806 04:38:28.663994 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:28.664049 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:28.665374 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:28.665908 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:28.665916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:28.666373 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:28.667049 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:28.667068 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.667890 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.667893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:28.668857 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:28.668868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.669365 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.669369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:28.669533 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:28.670133 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:28.670151 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:28.671855 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:28.674092 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:28.674098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:28.674113 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:28.674460 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:28.674835 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:28.674838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:28.674839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:28.675009 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:28.675512 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:28.675518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:28.675998 (+ 480us) client_negotiation.cc:769] Sending connection context
0806 04:38:28.676197 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 04:38:28.676553 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":123,"threads_started":1}
I20260806 04:38:28.679167 6381 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41007
I20260806 04:38:28.679415 6381 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:28.679977 6381 heartbeater.cc:511] Master 127.12.15.254:41007 requested a full tablet report, sending...
I20260806 04:38:28.681754 6054 ts_manager.cc:194] Registered new tserver with Master: 57bb2b06aa694b32a43f6c6976cf5ef4 (127.12.15.193:46425)
I20260806 04:38:28.683393 6054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:57015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:38:28.696537 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41007
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:28.819101 6387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:28.819486 6387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:28.819567 6387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:28.825731 6387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:28.825807 6387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:28.825898 6387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:28.833876 6387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:28.835319 6387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:28.836328 6387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:28.845860 6398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:28.845911 6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:28.845881 6401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:28.846052 6387 server_base.cc:1034] running on GCE node
I20260806 04:38:28.846524 6387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:28.847167 6387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:28.848366 6387 hybrid_clock.cc:648] HybridClock initialized: now 1785991108848316 us; error 32 us; skew 500 ppm
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:28.851121 6387 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:38:28.852015 6387 webserver.cc:460] Webserver started at http://127.12.15.194:44997/ using document root <none> and password file <none>
I20260806 04:38:28.852612 6387 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:28.852682 6387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:28.852890 6387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:28.855010 6387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "125d7357b7fb4781ac43ce0194950421"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "aac85651dd35eae9b9230d1a39498592"
server_key_iv: "d478f26c706152bf85de4753978637a1"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.855551 6387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "125d7357b7fb4781ac43ce0194950421"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "aac85651dd35eae9b9230d1a39498592"
server_key_iv: "d478f26c706152bf85de4753978637a1"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.859270 6387 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:38:28.861799 6411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.862999 6387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:28.863116 6387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "125d7357b7fb4781ac43ce0194950421"
format_stamp: "Formatted at 2026-08-06 04:38:28 on dist-test-slave-xt4k"
server_key: "aac85651dd35eae9b9230d1a39498592"
server_key_iv: "d478f26c706152bf85de4753978637a1"
server_key_version: "encryptionkey@0"
I20260806 04:38:28.863209 6387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:28.874238 6387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:28.877336 6387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:28.877506 6387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:28.878062 6387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:28.878947 6387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:28.878998 6387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.879055 6387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:28.879086 6387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:28.887440 6387 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:44583
I20260806 04:38:28.887480 6590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:44583 every 8 connection(s)
I20260806 04:38:28.888320 6387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:28.894043 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 6387
I20260806 04:38:28.894161 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:38:28.894454 12351 external_mini_cluster.cc:1428] Setting key 80e27c7bf71fc0c3930927301363afb8
Aug 06 04:38:28 dist-test-slave-xt4k krb5kdc[5985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:28.901865 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:28.890158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:43445 (local address 127.12.15.254:41007)
0806 04:38:28.890313 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:28.890317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:28.891151 (+ 834us) server_negotiation.cc:407] Connection header received
0806 04:38:28.892507 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:28.892510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:28.892584 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:28.892649 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:28.893785 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.894550 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.895955 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.896257 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.896417 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:28.898660 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:28.898663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:28.898665 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:28.898688 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:28.899921 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:28.900395 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:28.900399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:28.900401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:28.900495 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:28.900825 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:28.900828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:28.900829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:28.901003 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:28.901098 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:28.901580 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 04:38:28.901703 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:38:28.902689 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:28.890513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41007 (local address 127.12.15.194:43445)
0806 04:38:28.890992 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:28.891033 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:28.892262 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:28.892821 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:28.892828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:28.893182 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:28.893659 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:28.893672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.894670 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.894672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:28.895830 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:28.895841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:28.896417 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:28.896420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:28.896561 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:28.897101 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:28.897117 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:28.898495 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:28.900029 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:28.900036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:28.900051 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:28.900289 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:28.900590 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:28.900594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:28.900596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:28.900726 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:28.901099 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:28.901106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:28.901345 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 04:38:28.901501 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 04:38:28.901815 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":108,"threads_started":1}
I20260806 04:38:28.903759 6591 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41007
I20260806 04:38:28.903980 6591 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:38:28.904495 6591 heartbeater.cc:511] Master 127.12.15.254:41007 requested a full tablet report, sending...
I20260806 04:38:28.905548 6054 ts_manager.cc:194] Registered new tserver with Master: 125d7357b7fb4781ac43ce0194950421 (127.12.15.194:44583)
I20260806 04:38:28.906093 6054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:43445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:38:28.921272 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41007
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:29.036808 6599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:29.037067 6599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:29.037117 6599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:29.041121 6599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:29.041183 6599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:29.041241 6599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:29.046528 6599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:41007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:29.047639 6599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:29.048678 6599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:29.058382 6606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:29.058394 6607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:29.058394 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:29.058840 6599 server_base.cc:1034] running on GCE node
I20260806 04:38:29.059242 6599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:29.059912 6599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:29.061072 6599 hybrid_clock.cc:648] HybridClock initialized: now 1785991109061054 us; error 33 us; skew 500 ppm
Aug 06 04:38:29 dist-test-slave-xt4k krb5kdc[5985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:29.063875 6599 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:38:29.064850 6599 webserver.cc:460] Webserver started at http://127.12.15.195:36365/ using document root <none> and password file <none>
I20260806 04:38:29.065421 6599 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:29.065490 6599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:29.065698 6599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:29.067884 6599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "15ab54758eaf46419d094cd6745e53b3"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "0662f55db3930ec9c8b7223779e0dad6"
server_key_iv: "17848474f51cd014819df0b20453b197"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.068483 6599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "15ab54758eaf46419d094cd6745e53b3"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "0662f55db3930ec9c8b7223779e0dad6"
server_key_iv: "17848474f51cd014819df0b20453b197"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.072438 6599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:38:29.074985 6626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:29.076299 6599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:29.076440 6599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "15ab54758eaf46419d094cd6745e53b3"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "0662f55db3930ec9c8b7223779e0dad6"
server_key_iv: "17848474f51cd014819df0b20453b197"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.076539 6599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:29.092650 6599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:29.095743 6599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:29.095906 6599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:29.096535 6599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:29.097363 6599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:29.097410 6599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:29.097452 6599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:29.097472 6599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:29.106437 6599 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:42975
I20260806 04:38:29.106446 6798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:42975 every 8 connection(s)
I20260806 04:38:29.107393 6599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:38:29.108052 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 6599
I20260806 04:38:29.108259 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:38:29.108619 12351 external_mini_cluster.cc:1428] Setting key 2c48df7799b924e3e29d081d53caf0fc
Aug 06 04:38:29 dist-test-slave-xt4k krb5kdc[5985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:29.121361 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.109379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33173 (local address 127.12.15.254:41007)
0806 04:38:29.109536 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.109540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.110768 (+ 1228us) server_negotiation.cc:407] Connection header received
0806 04:38:29.112198 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.112200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.112249 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.112323 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:29.113524 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.114088 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.115376 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.115595 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.115683 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.118083 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:29.118086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:29.118088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:29.118124 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:29.119402 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:29.119875 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:29.119877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:29.119878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:29.119951 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:29.120232 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:29.120234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:29.120235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:29.120446 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:29.120521 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.121082 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.121218 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 04:38:29.122228 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.109946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:41007 (local address 127.12.15.195:33173)
0806 04:38:29.110481 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.110535 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:29.111989 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:29.112497 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:29.112504 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:29.112865 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:29.113372 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.113385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.114244 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.114247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.115249 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.115261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.115722 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.115724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.115859 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.116482 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:29.116499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:29.117919 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:29.119517 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:29.119523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:29.119536 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:29.119797 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:29.120028 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:29.120031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:29.120032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:29.120165 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:29.120526 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:29.120532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:29.120776 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 04:38:29.120962 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 04:38:29.121227 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":130,"threads_started":1}
I20260806 04:38:29.123612 6799 heartbeater.cc:347] Connected to a master server at 127.12.15.254:41007
I20260806 04:38:29.123870 6799 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:29.124440 6799 heartbeater.cc:511] Master 127.12.15.254:41007 requested a full tablet report, sending...
I20260806 04:38:29.125558 6054 ts_manager.cc:194] Registered new tserver with Master: 15ab54758eaf46419d094cd6745e53b3 (127.12.15.195:42975)
I20260806 04:38:29.126166 6054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:33173
I20260806 04:38:29.134948 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:38:29.144740 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.137396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.12.15.254:41007)
0806 04:38:29.137598 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.137602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.137646 (+ 44us) server_negotiation.cc:407] Connection header received
0806 04:38:29.137816 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.137819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.137865 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.137936 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:29.138637 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.139194 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.140058 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.140288 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.140418 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.141794 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:29.141798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:29.141799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:29.141823 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:29.143165 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:29.143571 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:29.143574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:29.143575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:29.143656 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:29.143878 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:29.143880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:29.143882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:29.144114 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:29.144220 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.144445 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.144576 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:38:29.147830 6054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:38:29.150938 6054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:38:29.165851 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.160526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:46425 (local address 127.0.0.1:46056)
0806 04:38:29.160860 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.160873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:29.160966 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:29.161969 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:29.161972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:29.161991 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:29.162081 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.162087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.163081 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.163085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.164331 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.164366 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.165313 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.165316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.165422 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.165459 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 04:38:29.165569 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 04:38:29.165700 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":89,"threads_started":1}
I20260806 04:38:29.165860 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.160884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.12.15.193:46425)
0806 04:38:29.161291 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.161310 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.161334 (+ 24us) server_negotiation.cc:407] Connection header received
0806 04:38:29.161410 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.161415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.161627 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.161788 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:29.162178 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.162954 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.164496 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.165198 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.165298 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.165456 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.165610 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.165708 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":88,"threads_started":1}
I20260806 04:38:29.166213 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.159894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42975 (local address 127.0.0.1:59130)
0806 04:38:29.160454 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.160511 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:29.160658 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:29.161840 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:29.161843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:29.161857 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:29.161928 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.161933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.163566 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.163569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.164555 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.164565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.165655 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.165659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.165761 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.165806 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 04:38:29.165852 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 04:38:29.166065 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":47,"spinlock_wait_cycles":15744,"thread_start_us":128,"threads_started":1}
I20260806 04:38:29.166410 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.159922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44583 (local address 127.0.0.1:39166)
0806 04:38:29.160524 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.160536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:29.160658 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:29.161712 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:29.161718 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:29.161742 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:29.161898 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.161905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.163539 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.163542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.164693 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.164702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.165706 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.165709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.166038 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.166080 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 04:38:29.166145 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:38:29.166203 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":229,"spinlock_wait_cycles":2688,"thread_start_us":102,"threads_started":1}
I20260806 04:38:29.166581 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.160848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39166 (local address 127.12.15.194:44583)
0806 04:38:29.161191 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.161203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.161230 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:38:29.161314 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.161320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.161535 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.162071 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:29.162211 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.163390 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.164822 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.165597 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.165824 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.165968 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.166325 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.166442 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":121,"threads_started":1}
I20260806 04:38:29.167037 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.160055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.12.15.195:42975)
0806 04:38:29.161441 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.161452 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.161472 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:38:29.161540 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.161544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.161706 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.162062 (+ 356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:29.162200 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.163390 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.164684 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.165513 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.166559 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.166754 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.166804 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.166893 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1281,"thread_start_us":163,"threads_started":1}
I20260806 04:38:29.169492 6702 tablet_service.cc:1467] Processing CreateTablet for tablet f8cafec75afd4d6996e0693730376d29 (DEFAULT_TABLE table=test-table [id=6dcb10ec0e1947c3a2e882ca08bc9818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:29.169399 6491 tablet_service.cc:1467] Processing CreateTablet for tablet f8cafec75afd4d6996e0693730376d29 (DEFAULT_TABLE table=test-table [id=6dcb10ec0e1947c3a2e882ca08bc9818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:29.169400 6281 tablet_service.cc:1467] Processing CreateTablet for tablet f8cafec75afd4d6996e0693730376d29 (DEFAULT_TABLE table=test-table [id=6dcb10ec0e1947c3a2e882ca08bc9818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:29.170429 6702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8cafec75afd4d6996e0693730376d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:29.170457 6281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8cafec75afd4d6996e0693730376d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:29.170457 6491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8cafec75afd4d6996e0693730376d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:29.177106 6825 tablet_bootstrap.cc:492] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Bootstrap starting.
I20260806 04:38:29.178083 6828 tablet_bootstrap.cc:492] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Bootstrap starting.
I20260806 04:38:29.179608 6825 tablet_bootstrap.cc:654] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:29.180019 6829 tablet_bootstrap.cc:492] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: Bootstrap starting.
I20260806 04:38:29.180285 6828 tablet_bootstrap.cc:654] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:29.181063 6825 log.cc:824] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:29.181550 6828 log.cc:824] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:29.181967 6829 tablet_bootstrap.cc:654] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:29.182729 6829 log.cc:824] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:29.183558 6825 tablet_bootstrap.cc:492] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: No bootstrap required, opened a new log
I20260806 04:38:29.183558 6828 tablet_bootstrap.cc:492] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: No bootstrap required, opened a new log
I20260806 04:38:29.183751 6825 ts_tablet_manager.cc:1397] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 04:38:29.183751 6828 ts_tablet_manager.cc:1397] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260806 04:38:29.184381 6829 tablet_bootstrap.cc:492] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: No bootstrap required, opened a new log
I20260806 04:38:29.184557 6829 ts_tablet_manager.cc:1397] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:38:29.187619 6828 raft_consensus.cc:348] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.187619 6825 raft_consensus.cc:348] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.187733 6829 raft_consensus.cc:348] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.187894 6825 raft_consensus.cc:374] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:29.187894 6828 raft_consensus.cc:374] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:29.187930 6829 raft_consensus.cc:374] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:29.187953 6828 raft_consensus.cc:729] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125d7357b7fb4781ac43ce0194950421, State: Initialized, Role: FOLLOWER
I20260806 04:38:29.187953 6825 raft_consensus.cc:729] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ab54758eaf46419d094cd6745e53b3, State: Initialized, Role: FOLLOWER
I20260806 04:38:29.187973 6829 raft_consensus.cc:729] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bb2b06aa694b32a43f6c6976cf5ef4, State: Initialized, Role: FOLLOWER
I20260806 04:38:29.188438 6828 consensus_queue.cc:260] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.188460 6829 consensus_queue.cc:260] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.188443 6825 consensus_queue.cc:260] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.189448 6825 ts_tablet_manager.cc:1428] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 04:38:29.189438 6828 ts_tablet_manager.cc:1428] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 04:38:29.189436 6829 ts_tablet_manager.cc:1428] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 04:38:29.189886 6591 heartbeater.cc:503] Master 127.12.15.254:41007 was elected leader, sending a full tablet report...
I20260806 04:38:29.190021 6799 heartbeater.cc:503] Master 127.12.15.254:41007 was elected leader, sending a full tablet report...
I20260806 04:38:29.190083 6381 heartbeater.cc:503] Master 127.12.15.254:41007 was elected leader, sending a full tablet report...
I20260806 04:38:29.340106 6840 raft_consensus.cc:482] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:29.340373 6840 raft_consensus.cc:504] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.341568 6840 leader_election.cc:288] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15ab54758eaf46419d094cd6745e53b3 (127.12.15.195:42975), 125d7357b7fb4781ac43ce0194950421 (127.12.15.194:44583)
I20260806 04:38:29.346992 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.341993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:33941 (local address 127.12.15.194:44583)
0806 04:38:29.342107 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.342112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.342421 (+ 309us) server_negotiation.cc:407] Connection header received
0806 04:38:29.342565 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.342568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.342626 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.342725 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:29.343059 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.343844 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.345703 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.346362 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.346502 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.346545 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.346767 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.346852 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:38:29.347012 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.341874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44583 (local address 127.12.15.193:33941)
0806 04:38:29.342275 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.342290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:29.342446 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:29.342732 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:29.342736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:29.342767 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:29.342943 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.342951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.343982 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.343986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.345567 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.345575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.346520 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.346524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.346626 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.346652 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:38:29.346736 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:38:29.346818 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":169,"thread_start_us":118,"threads_started":1}
I20260806 04:38:29.347096 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.341959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:36027 (local address 127.12.15.195:42975)
0806 04:38:29.342099 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.342104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.342438 (+ 334us) server_negotiation.cc:407] Connection header received
0806 04:38:29.342565 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.342568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.342628 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.342718 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:29.343059 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.343845 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.345703 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.346342 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.346477 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.346538 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.346887 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.346975 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 04:38:29.347591 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.341876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:42975 (local address 127.12.15.193:36027)
0806 04:38:29.342317 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.342328 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:29.342445 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:29.342732 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:29.342736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:29.342780 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:29.342958 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.342965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.343982 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.343986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.345553 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:29.345567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.346631 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.346635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:29.346739 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.346765 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:38:29.347352 (+ 587us) client_negotiation.cc:240] Negotiation successful
0806 04:38:29.347434 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":158,"spinlock_wait_cycles":45568,"thread_start_us":85,"threads_started":1}
I20260806 04:38:29.347851 6525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8cafec75afd4d6996e0693730376d29" candidate_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125d7357b7fb4781ac43ce0194950421" is_pre_election: true
I20260806 04:38:29.348205 6525 raft_consensus.cc:2454] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb2b06aa694b32a43f6c6976cf5ef4 in term 0.
I20260806 04:38:29.348299 6728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8cafec75afd4d6996e0693730376d29" candidate_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15ab54758eaf46419d094cd6745e53b3" is_pre_election: true
I20260806 04:38:29.348618 6728 raft_consensus.cc:2454] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb2b06aa694b32a43f6c6976cf5ef4 in term 0.
I20260806 04:38:29.348779 6194 leader_election.cc:302] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125d7357b7fb4781ac43ce0194950421, 57bb2b06aa694b32a43f6c6976cf5ef4; no voters:
I20260806 04:38:29.349112 6840 raft_consensus.cc:2781] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:38:29.349202 6840 raft_consensus.cc:482] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:38:29.349253 6840 raft_consensus.cc:3037] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:29.350294 6840 raft_consensus.cc:504] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.350658 6840 leader_election.cc:288] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [CANDIDATE]: Term 1 election: Requested vote from peers 15ab54758eaf46419d094cd6745e53b3 (127.12.15.195:42975), 125d7357b7fb4781ac43ce0194950421 (127.12.15.194:44583)
I20260806 04:38:29.350998 6728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8cafec75afd4d6996e0693730376d29" candidate_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15ab54758eaf46419d094cd6745e53b3"
I20260806 04:38:29.351039 6525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8cafec75afd4d6996e0693730376d29" candidate_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125d7357b7fb4781ac43ce0194950421"
I20260806 04:38:29.351141 6728 raft_consensus.cc:3037] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:29.351141 6525 raft_consensus.cc:3037] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:29.352406 6525 raft_consensus.cc:2454] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb2b06aa694b32a43f6c6976cf5ef4 in term 1.
I20260806 04:38:29.352406 6728 raft_consensus.cc:2454] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb2b06aa694b32a43f6c6976cf5ef4 in term 1.
I20260806 04:38:29.352722 6194 leader_election.cc:302] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125d7357b7fb4781ac43ce0194950421, 57bb2b06aa694b32a43f6c6976cf5ef4; no voters:
I20260806 04:38:29.352890 6840 raft_consensus.cc:2781] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:29.353133 6840 raft_consensus.cc:686] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 1 LEADER]: Becoming Leader. State: Replica: 57bb2b06aa694b32a43f6c6976cf5ef4, State: Running, Role: LEADER
I20260806 04:38:29.353480 6840 consensus_queue.cc:237] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } }
I20260806 04:38:29.356534 6053 catalog_manager.cc:5495] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57bb2b06aa694b32a43f6c6976cf5ef4 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb2b06aa694b32a43f6c6976cf5ef4" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 46425 } health_report { overall_health: HEALTHY } } }
W20260806 04:38:29.358559 6801 tablet.cc:2369] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:38:29.389605 6592 tablet.cc:2369] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:29.401515 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.398260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.12.15.193:46425)
0806 04:38:29.398392 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.398396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.398409 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:38:29.398504 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.398507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.398567 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.398653 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:38:29.398845 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.399346 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.400127 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.400384 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.400477 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.400734 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:38:29.401166 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:38:29.401257 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.401321 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.401373 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:38:29.407692 6525 raft_consensus.cc:1261] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 1 FOLLOWER]: Refusing update from remote peer 57bb2b06aa694b32a43f6c6976cf5ef4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:29.407748 6728 raft_consensus.cc:1261] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 1 FOLLOWER]: Refusing update from remote peer 57bb2b06aa694b32a43f6c6976cf5ef4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:29.408390 6840 consensus_queue.cc:1035] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "125d7357b7fb4781ac43ce0194950421" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:29.408596 6845 consensus_queue.cc:1035] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15ab54758eaf46419d094cd6745e53b3" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 42975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:38:29.412377 6383 tablet.cc:2369] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:29.417196 6854 mvcc.cc:205] Tried to move back new op lower bound from 7315419584125566976 to 7315419583914364928. Current Snapshot: MvccSnapshot[applied={T|T < 7315419584125566976}]
I20260806 04:38:29.419014 6853 mvcc.cc:205] Tried to move back new op lower bound from 7315419584125566976 to 7315419583914364928. Current Snapshot: MvccSnapshot[applied={T|T < 7315419584125566976}]
I20260806 04:38:29.425254 6053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:29.425464 6053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:29.428079 6053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 371615d0519243bf9c56b1fb2391b7a9: Sending DeleteTablet for 3 replicas of tablet f8cafec75afd4d6996e0693730376d29
I20260806 04:38:29.428948 6491 tablet_service.cc:1514] Processing DeleteTablet for tablet f8cafec75afd4d6996e0693730376d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:29 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:39166
I20260806 04:38:29.428999 6702 tablet_service.cc:1514] Processing DeleteTablet for tablet f8cafec75afd4d6996e0693730376d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:29 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:59130
I20260806 04:38:29.429455 6866 tablet_replica.cc:330] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: stopping tablet replica
I20260806 04:38:29.429533 6868 tablet_replica.cc:330] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: stopping tablet replica
I20260806 04:38:29.429785 6281 tablet_service.cc:1514] Processing DeleteTablet for tablet f8cafec75afd4d6996e0693730376d29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:29 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:46056
I20260806 04:38:29.429876 6868 raft_consensus.cc:2229] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:29.429878 6866 raft_consensus.cc:2229] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:29.430478 6869 tablet_replica.cc:330] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4: stopping tablet replica
I20260806 04:38:29.430486 6866 raft_consensus.cc:2258] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:29.430673 6868 raft_consensus.cc:2258] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:29.430830 6869 raft_consensus.cc:2229] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:38:29.431171 6869 raft_consensus.cc:2258] T f8cafec75afd4d6996e0693730376d29 P 57bb2b06aa694b32a43f6c6976cf5ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:29.431329 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 6167
I20260806 04:38:29.432657 6868 ts_tablet_manager.cc:1905] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:29.435195 6868 ts_tablet_manager.cc:1918] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:29.435273 6868 log.cc:1196] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/f8cafec75afd4d6996e0693730376d29
I20260806 04:38:29.435491 6868 ts_tablet_manager.cc:1939] T f8cafec75afd4d6996e0693730376d29 P 15ab54758eaf46419d094cd6745e53b3: Deleting consensus metadata
I20260806 04:38:29.437076 6036 catalog_manager.cc:4841] TS 15ab54758eaf46419d094cd6745e53b3 (127.12.15.195:42975): tablet f8cafec75afd4d6996e0693730376d29 (table test-table [id=6dcb10ec0e1947c3a2e882ca08bc9818]) successfully deleted
I20260806 04:38:29.439499 6866 ts_tablet_manager.cc:1905] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:29.441790 6866 ts_tablet_manager.cc:1918] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:29.441888 6866 log.cc:1196] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/f8cafec75afd4d6996e0693730376d29
I20260806 04:38:29.442147 6866 ts_tablet_manager.cc:1939] T f8cafec75afd4d6996e0693730376d29 P 125d7357b7fb4781ac43ce0194950421: Deleting consensus metadata
I20260806 04:38:29.443351 6034 catalog_manager.cc:4841] TS 125d7357b7fb4781ac43ce0194950421 (127.12.15.194:44583): tablet f8cafec75afd4d6996e0693730376d29 (table test-table [id=6dcb10ec0e1947c3a2e882ca08bc9818]) successfully deleted
W20260806 04:38:29.444391 6038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:46425 (error 108)
I20260806 04:38:29.445233 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 6387
I20260806 04:38:29.445636 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.445265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:46425 (local address 127.0.0.1:46070)
0806 04:38:29.445469 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:29.445547 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 04:38:29.470641 6038 catalog_manager.cc:4584] TS 57bb2b06aa694b32a43f6c6976cf5ef4 (127.12.15.193:46425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8cafec75afd4d6996e0693730376d29: Network error: Client connection negotiation failed: client connection to 127.12.15.193:46425: connect: Connection refused (error 111)
I20260806 04:38:29.476218 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 6599
I20260806 04:38:29.486752 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 6007
2026-08-06T04:38:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3560 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T04:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:29Z Disabled control of system clock
I20260806 04:38:29.526582 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:35951
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:35951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 04:38:29.641978 6876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:29.642233 6876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:29.642266 6876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:29.646173 6876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:29.646227 6876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:29.646241 6876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:29.646256 6876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:29.646270 6876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:29.650940 6876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:35951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:35951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:29.651961 6876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:29.653349 6876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:29.661607 6887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:29.661648 6890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:29.661780 6876 server_base.cc:1034] running on GCE node
W20260806 04:38:29.661653 6885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:29.662375 6876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:29.663461 6876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:29.664615 6876 hybrid_clock.cc:648] HybridClock initialized: now 1785991109664587 us; error 39 us; skew 500 ppm
I20260806 04:38:29.666419 6876 webserver.cc:460] Webserver started at http://127.12.15.254:38797/ using document root <none> and password file <none>
I20260806 04:38:29.666986 6876 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:29.667052 6876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:29.667271 6876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:29.669359 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "1281b5ef5110467a8f500bf784a2be5f"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "fc4a209fff83ccb85e59f8e6e9c121b2"
server_key_iv: "d1e0f15d594a2c1351f384e59c9e996d"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.669925 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "1281b5ef5110467a8f500bf784a2be5f"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "fc4a209fff83ccb85e59f8e6e9c121b2"
server_key_iv: "d1e0f15d594a2c1351f384e59c9e996d"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.673539 6876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:38:29.675896 6897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:29.677235 6876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:29.677371 6876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "1281b5ef5110467a8f500bf784a2be5f"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "fc4a209fff83ccb85e59f8e6e9c121b2"
server_key_iv: "d1e0f15d594a2c1351f384e59c9e996d"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.677474 6876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:29.689023 6876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:29.689692 6876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:29.689865 6876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:29.696834 6876 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:35951
I20260806 04:38:29.696879 6976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:35951 every 8 connection(s)
I20260806 04:38:29.697818 6876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:29.701094 6979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:29.702929 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 6876
I20260806 04:38:29.703079 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:29.703372 12351 external_mini_cluster.cc:1428] Setting key d6600ab5d5a9e6927473d2ccc3eb0b98
I20260806 04:38:29.708448 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f: Bootstrap starting.
I20260806 04:38:29.710740 6979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:29.711481 6979 log.cc:824] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:29.712636 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:29.704865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.12.15.254:35951)
0806 04:38:29.705462 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:29.705477 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:29.705512 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:38:29.706191 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:29.706209 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:29.706613 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:29.706917 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:29.707470 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.708480 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.709379 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:29.709845 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:29.709997 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:29.710766 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:29.710771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:29.710785 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:29.711186 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:29.711245 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:29.711266 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:29.711352 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:29.711491 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 04:38:29.711871 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260806 04:38:29.713469 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f: No bootstrap required, opened a new log
I20260806 04:38:29.717584 6979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281b5ef5110467a8f500bf784a2be5f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35951 } }
I20260806 04:38:29.717823 6979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:29.717882 6979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1281b5ef5110467a8f500bf784a2be5f, State: Initialized, Role: FOLLOWER
I20260806 04:38:29.718319 6979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281b5ef5110467a8f500bf784a2be5f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35951 } }
I20260806 04:38:29.718448 6979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:29.718499 6979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:29.718578 6979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:29.719641 6979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281b5ef5110467a8f500bf784a2be5f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35951 } }
I20260806 04:38:29.720002 6979 leader_election.cc:302] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1281b5ef5110467a8f500bf784a2be5f; no voters:
I20260806 04:38:29.720317 6979 leader_election.cc:288] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:29.720460 6986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:29.720697 6986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [term 1 LEADER]: Becoming Leader. State: Replica: 1281b5ef5110467a8f500bf784a2be5f, State: Running, Role: LEADER
I20260806 04:38:29.721019 6986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281b5ef5110467a8f500bf784a2be5f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35951 } }
I20260806 04:38:29.721562 6979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:29.722587 6989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1281b5ef5110467a8f500bf784a2be5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281b5ef5110467a8f500bf784a2be5f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35951 } } }
I20260806 04:38:29.722724 6989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:29.722994 6990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1281b5ef5110467a8f500bf784a2be5f. Latest consensus state: current_term: 1 leader_uuid: "1281b5ef5110467a8f500bf784a2be5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1281b5ef5110467a8f500bf784a2be5f" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35951 } } }
I20260806 04:38:29.723090 6990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:29.723208 7007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:29.726548 7007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:29.732438 7007 catalog_manager.cc:1347] Generated new cluster ID: adb413a571374d06b9c3c12b0ca73bc8
I20260806 04:38:29.732511 7007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:29.742589 7007 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:29.744010 7007 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:29.748713 7007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f: Generated new TSK 0
I20260806 04:38:29.749626 7007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:29.765298 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:35951
--builtin_ntp_servers=127.12.15.212:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 04:38:29.924829 7020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:29.925127 7020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:29.925165 7020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:29.929203 7020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:29.929255 7020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:29.929308 7020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:29.934024 7020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:29.935213 7020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:29.936966 7020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:29.947679 7029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:29.947760 7020 server_base.cc:1034] running on GCE node
W20260806 04:38:29.947726 7032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:29.947696 7030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:29.948326 7020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:29.949019 7020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:29.950212 7020 hybrid_clock.cc:648] HybridClock initialized: now 1785991109950189 us; error 31 us; skew 500 ppm
I20260806 04:38:29.952154 7020 webserver.cc:460] Webserver started at http://127.12.15.193:33523/ using document root <none> and password file <none>
I20260806 04:38:29.952783 7020 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:29.952859 7020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:29.953068 7020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:29.955178 7020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "37be2c449cee49eeaefe4e51a6c2780b"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "1df5c656c8aa5ed452f48800a90fc65b"
server_key_iv: "b2c1cd3ca24f97f23b71d509f4ba2977"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.955740 7020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "37be2c449cee49eeaefe4e51a6c2780b"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "1df5c656c8aa5ed452f48800a90fc65b"
server_key_iv: "b2c1cd3ca24f97f23b71d509f4ba2977"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.959549 7020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:38:29.962198 7042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:29.963512 7020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:29.963668 7020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "37be2c449cee49eeaefe4e51a6c2780b"
format_stamp: "Formatted at 2026-08-06 04:38:29 on dist-test-slave-xt4k"
server_key: "1df5c656c8aa5ed452f48800a90fc65b"
server_key_iv: "b2c1cd3ca24f97f23b71d509f4ba2977"
server_key_version: "encryptionkey@0"
I20260806 04:38:29.963778 7020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:29.991005 7020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:29.991797 7020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:29.991992 7020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:29.992678 7020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:29.993697 7020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:29.993748 7020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:29.993806 7020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:29.993834 7020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.002538 7020 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:34347
I20260806 04:38:30.002635 7237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:34347 every 8 connection(s)
I20260806 04:38:30.003501 7020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:30.008020 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 7020
I20260806 04:38:30.008167 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:38:30.008517 12351 external_mini_cluster.cc:1428] Setting key 37dfec7ce28074fe78dea22a8325ec71
I20260806 04:38:30.010789 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:35951
--builtin_ntp_servers=127.12.15.212:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 04:38:30.014012 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.005420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:40307 (local address 127.12.15.254:35951)
0806 04:38:30.005589 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.005593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.006647 (+ 1054us) server_negotiation.cc:407] Connection header received
0806 04:38:30.008169 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.008172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.008245 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.008330 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:30.009512 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.010245 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.011807 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.012036 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.012144 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.013488 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:30.013492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:30.013495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:30.013511 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:30.013536 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:30.013547 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:30.013626 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.013777 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.013853 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 04:38:30.014919 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.005799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:35951 (local address 127.12.15.193:40307)
0806 04:38:30.006481 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.006524 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.007893 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.008563 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.008573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.008689 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:30.009364 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.009381 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.010378 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.010382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.011659 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.011672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.012210 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.012214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.012442 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.013174 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:30.013176 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:30.013222 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:30.013312 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:30.013631 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:30.013639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:30.013645 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.013762 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.014056 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":113,"threads_started":1}
I20260806 04:38:30.016212 7238 heartbeater.cc:347] Connected to a master server at 127.12.15.254:35951
I20260806 04:38:30.016518 7238 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:30.017227 7238 heartbeater.cc:511] Master 127.12.15.254:35951 requested a full tablet report, sending...
I20260806 04:38:30.018904 6920 ts_manager.cc:194] Registered new tserver with Master: 37be2c449cee49eeaefe4e51a6c2780b (127.12.15.193:34347)
I20260806 04:38:30.020080 6920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.193:40307
W20260806 04:38:30.127300 7243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:30.127562 7243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:30.127602 7243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:30.131942 7243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:30.131995 7243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:30.132055 7243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:30.136847 7243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:30.137959 7243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:30.139353 7243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:30.149534 7256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:30.149708 7243 server_base.cc:1034] running on GCE node
W20260806 04:38:30.149555 7253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:30.149555 7251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:30.150276 7243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:30.150905 7243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:30.152076 7243 hybrid_clock.cc:648] HybridClock initialized: now 1785991110152054 us; error 30 us; skew 500 ppm
I20260806 04:38:30.153930 7243 webserver.cc:460] Webserver started at http://127.12.15.194:38111/ using document root <none> and password file <none>
I20260806 04:38:30.154512 7243 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:30.154574 7243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:30.154788 7243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:30.156930 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "5496e7c1cf7c46d58b3560af28d918ec"
format_stamp: "Formatted at 2026-08-06 04:38:30 on dist-test-slave-xt4k"
server_key: "e17b4977c5e6547054d96564244fea28"
server_key_iv: "3da0d8bb5547df1b57544c57a6b64471"
server_key_version: "encryptionkey@0"
I20260806 04:38:30.157485 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "5496e7c1cf7c46d58b3560af28d918ec"
format_stamp: "Formatted at 2026-08-06 04:38:30 on dist-test-slave-xt4k"
server_key: "e17b4977c5e6547054d96564244fea28"
server_key_iv: "3da0d8bb5547df1b57544c57a6b64471"
server_key_version: "encryptionkey@0"
I20260806 04:38:30.161298 7243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:38:30.163741 7265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.164968 7243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:30.165077 7243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "5496e7c1cf7c46d58b3560af28d918ec"
format_stamp: "Formatted at 2026-08-06 04:38:30 on dist-test-slave-xt4k"
server_key: "e17b4977c5e6547054d96564244fea28"
server_key_iv: "3da0d8bb5547df1b57544c57a6b64471"
server_key_version: "encryptionkey@0"
I20260806 04:38:30.165185 7243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:30.177978 7243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:30.178632 7243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:30.178793 7243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:30.179347 7243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:30.180262 7243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:30.180320 7243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.180395 7243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:30.180423 7243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.188752 7243 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:33517
I20260806 04:38:30.188808 7440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:33517 every 8 connection(s)
I20260806 04:38:30.189625 7243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:30.198436 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.191372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:56757 (local address 127.12.15.254:35951)
0806 04:38:30.191506 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.191510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.192613 (+ 1103us) server_negotiation.cc:407] Connection header received
0806 04:38:30.193980 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.193983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.194042 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.194118 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:30.195057 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.195590 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.196724 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.196943 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.197029 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.197982 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:30.197986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:30.197988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:30.198001 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:30.198021 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:30.198029 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:30.198089 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.198233 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.198292 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:38:30.198879 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 7243
I20260806 04:38:30.198997 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:38:30.199271 12351 external_mini_cluster.cc:1428] Setting key cb51635defcc7e5a7ef34f4e0e65c002
I20260806 04:38:30.199285 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.191759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:35951 (local address 127.12.15.194:56757)
0806 04:38:30.192469 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.192519 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.193777 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.194291 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.194298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.194384 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:30.194918 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.194931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.195703 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.195707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.196613 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.196623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.197080 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.197085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.197229 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.197749 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:30.197751 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:30.197786 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:30.197845 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:30.198098 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:30.198107 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:30.198114 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.198218 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.198476 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":104,"threads_started":1}
I20260806 04:38:30.200372 7441 heartbeater.cc:347] Connected to a master server at 127.12.15.254:35951
I20260806 04:38:30.200587 7441 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:30.201169 7441 heartbeater.cc:511] Master 127.12.15.254:35951 requested a full tablet report, sending...
I20260806 04:38:30.201357 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:35951
--builtin_ntp_servers=127.12.15.212:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 04:38:30.202208 6919 ts_manager.cc:194] Registered new tserver with Master: 5496e7c1cf7c46d58b3560af28d918ec (127.12.15.194:33517)
I20260806 04:38:30.202790 6919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.194:56757
W20260806 04:38:30.312888 7444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:30.313145 7444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:30.313184 7444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:30.317176 7444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:30.317235 7444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:30.317296 7444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:30.322048 7444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:30.323122 7444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:30.324538 7444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:30.333853 7453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:30.333913 7454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:30.333864 7460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:30.334184 7444 server_base.cc:1034] running on GCE node
I20260806 04:38:30.334540 7444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:30.335131 7444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:30.336279 7444 hybrid_clock.cc:648] HybridClock initialized: now 1785991110336262 us; error 30 us; skew 500 ppm
I20260806 04:38:30.338033 7444 webserver.cc:460] Webserver started at http://127.12.15.195:36517/ using document root <none> and password file <none>
I20260806 04:38:30.338578 7444 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:30.338645 7444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:30.338845 7444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:30.340859 7444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "cdbcea98d648450a8750e67fe19acf07"
format_stamp: "Formatted at 2026-08-06 04:38:30 on dist-test-slave-xt4k"
server_key: "5ebb854d5e1f5117d2dd53de17563ec4"
server_key_iv: "35bd9d6a5313927af2f869502dcd380c"
server_key_version: "encryptionkey@0"
I20260806 04:38:30.341390 7444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "cdbcea98d648450a8750e67fe19acf07"
format_stamp: "Formatted at 2026-08-06 04:38:30 on dist-test-slave-xt4k"
server_key: "5ebb854d5e1f5117d2dd53de17563ec4"
server_key_iv: "35bd9d6a5313927af2f869502dcd380c"
server_key_version: "encryptionkey@0"
I20260806 04:38:30.344952 7444 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:38:30.347447 7466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.348685 7444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:30.348790 7444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "cdbcea98d648450a8750e67fe19acf07"
format_stamp: "Formatted at 2026-08-06 04:38:30 on dist-test-slave-xt4k"
server_key: "5ebb854d5e1f5117d2dd53de17563ec4"
server_key_iv: "35bd9d6a5313927af2f869502dcd380c"
server_key_version: "encryptionkey@0"
I20260806 04:38:30.348894 7444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:30.368443 7444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:30.369226 7444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:30.369405 7444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:30.370029 7444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:30.371009 7444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:30.371054 7444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.371096 7444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:30.371127 7444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:30.379854 7444 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:44497
I20260806 04:38:30.379936 7635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:44497 every 8 connection(s)
I20260806 04:38:30.380870 7444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:38:30.388533 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 7444
I20260806 04:38:30.388670 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:38:30.388974 12351 external_mini_cluster.cc:1428] Setting key 7491af6774357b3df8f779f43d7c14ee
I20260806 04:38:30.390365 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.382836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:55395 (local address 127.12.15.254:35951)
0806 04:38:30.382959 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.382963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.384066 (+ 1103us) server_negotiation.cc:407] Connection header received
0806 04:38:30.385471 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.385474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.385527 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.385607 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:30.386522 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.387097 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.388289 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.388630 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.388769 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.389892 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:30.389895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:30.389897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:30.389909 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:30.389931 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:30.389939 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:30.389998 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.390153 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.390220 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 04:38:30.391291 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.383206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:35951 (local address 127.12.15.195:55395)
0806 04:38:30.383901 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.383941 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.385248 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.385774 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.385780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.385863 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:30.386361 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.386373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.387223 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.387226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.388117 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.388128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.388829 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.388832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.388986 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.389619 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:30.389621 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:30.389662 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:30.389729 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:30.390011 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:30.390020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:30.390026 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.390140 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.390429 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":108,"threads_started":1}
I20260806 04:38:30.392395 7636 heartbeater.cc:347] Connected to a master server at 127.12.15.254:35951
I20260806 04:38:30.392642 7636 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:30.393204 7636 heartbeater.cc:511] Master 127.12.15.254:35951 requested a full tablet report, sending...
I20260806 04:38:30.394325 6919 ts_manager.cc:194] Registered new tserver with Master: cdbcea98d648450a8750e67fe19acf07 (127.12.15.195:44497)
I20260806 04:38:30.395079 6919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.195:55395
I20260806 04:38:30.402910 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:38:30.409013 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.405150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.12.15.254:35951)
0806 04:38:30.405338 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.405343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.405503 (+ 160us) server_negotiation.cc:407] Connection header received
0806 04:38:30.405638 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.405641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.405690 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.405769 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:30.406068 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.406595 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.407430 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.407660 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.407750 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.408583 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:30.408586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:30.408588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:30.408601 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:30.408622 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:30.408630 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:30.408691 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.408785 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.408854 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 04:38:30.412568 6919 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:38:30.417361 6919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:38:30.435516 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.428290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:44497 (local address 127.0.0.1:36082)
0806 04:38:30.429473 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.429557 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.429727 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.430509 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.430515 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.430537 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:30.430669 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.430676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.432167 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.432170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.433647 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.433659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.434745 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.434749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.434887 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.434950 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.435085 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.435292 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"spinlock_wait_cycles":13696,"thread_start_us":129,"threads_started":1}
I20260806 04:38:30.435621 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.428627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:33517 (local address 127.0.0.1:39330)
0806 04:38:30.430139 (+ 1512us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.430153 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.430231 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.431108 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.431113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.431147 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:30.431341 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.431351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.432984 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.432987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.433952 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.433962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.435075 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.435079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.435186 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.435219 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.435287 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.435394 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1457,"thread_start_us":178,"threads_started":1}
I20260806 04:38:30.435608 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.429270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39330 (local address 127.12.15.194:33517)
0806 04:38:30.429722 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.429796 (+ 74us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.430315 (+ 519us) server_negotiation.cc:407] Connection header received
0806 04:38:30.430393 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.430397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.430564 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.430698 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:30.431569 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.432826 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.434125 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.434945 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.435067 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.435278 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.435336 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.435448 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":123,"threads_started":1}
I20260806 04:38:30.436398 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.428459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.12.15.195:44497)
0806 04:38:30.430052 (+ 1593us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.430063 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.430085 (+ 22us) server_negotiation.cc:407] Connection header received
0806 04:38:30.430162 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.430167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.430364 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.430506 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:30.430855 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.432025 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.433818 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.434599 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.434722 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.434954 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.436088 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.436202 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1494,"thread_start_us":172,"threads_started":1}
I20260806 04:38:30.437007 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.428306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:34347 (local address 127.0.0.1:50720)
0806 04:38:30.429473 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.429551 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.429708 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.432112 (+ 2404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.432116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.432139 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:30.432467 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.432478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.433958 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.433961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.434760 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.434770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.436587 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.436591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.436693 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.436722 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.436774 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.436860 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"spinlock_wait_cycles":7552,"thread_start_us":113,"threads_started":1}
I20260806 04:38:30.437741 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.429314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.12.15.193:34347)
0806 04:38:30.430033 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.430046 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.430075 (+ 29us) server_negotiation.cc:407] Connection header received
0806 04:38:30.430153 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.430160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.430374 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.432179 (+ 1805us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:30.432647 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.433835 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.434932 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.436414 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.437203 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.437404 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.437459 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.437584 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":102,"threads_started":1}
I20260806 04:38:30.439332 7339 tablet_service.cc:1467] Processing CreateTablet for tablet ae3a7e0e37ad4be9b8516c150d74c4df (DEFAULT_TABLE table=test-table [id=0eddde2a799e4ed58f6cfbab0c40053c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:30.440425 7339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3a7e0e37ad4be9b8516c150d74c4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:30.442906 7543 tablet_service.cc:1467] Processing CreateTablet for tablet ae3a7e0e37ad4be9b8516c150d74c4df (DEFAULT_TABLE table=test-table [id=0eddde2a799e4ed58f6cfbab0c40053c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:30.444082 7543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3a7e0e37ad4be9b8516c150d74c4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:30.446416 7131 tablet_service.cc:1467] Processing CreateTablet for tablet ae3a7e0e37ad4be9b8516c150d74c4df (DEFAULT_TABLE table=test-table [id=0eddde2a799e4ed58f6cfbab0c40053c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:30.447360 7131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae3a7e0e37ad4be9b8516c150d74c4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:30.448391 7659 tablet_bootstrap.cc:492] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Bootstrap starting.
I20260806 04:38:30.451015 7659 tablet_bootstrap.cc:654] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:30.452011 7659 log.cc:824] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:30.452061 7662 tablet_bootstrap.cc:492] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Bootstrap starting.
I20260806 04:38:30.454134 7662 tablet_bootstrap.cc:654] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:30.454504 7659 tablet_bootstrap.cc:492] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: No bootstrap required, opened a new log
I20260806 04:38:30.454752 7659 ts_tablet_manager.cc:1397] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 04:38:30.454900 7664 tablet_bootstrap.cc:492] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: Bootstrap starting.
I20260806 04:38:30.454941 7662 log.cc:824] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:30.456940 7664 tablet_bootstrap.cc:654] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:30.457013 7662 tablet_bootstrap.cc:492] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: No bootstrap required, opened a new log
I20260806 04:38:30.457226 7662 ts_tablet_manager.cc:1397] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260806 04:38:30.457621 7664 log.cc:824] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:30.459173 7659 raft_consensus.cc:348] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.459466 7659 raft_consensus.cc:374] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:30.459525 7659 raft_consensus.cc:729] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5496e7c1cf7c46d58b3560af28d918ec, State: Initialized, Role: FOLLOWER
I20260806 04:38:30.460089 7664 tablet_bootstrap.cc:492] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: No bootstrap required, opened a new log
I20260806 04:38:30.460116 7659 consensus_queue.cc:260] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.460275 7664 ts_tablet_manager.cc:1397] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:38:30.461150 7441 heartbeater.cc:503] Master 127.12.15.254:35951 was elected leader, sending a full tablet report...
I20260806 04:38:30.461228 7659 ts_tablet_manager.cc:1428] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 04:38:30.461525 7662 raft_consensus.cc:348] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.461751 7662 raft_consensus.cc:374] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:30.461802 7662 raft_consensus.cc:729] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdbcea98d648450a8750e67fe19acf07, State: Initialized, Role: FOLLOWER
I20260806 04:38:30.462842 7662 consensus_queue.cc:260] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.463805 7636 heartbeater.cc:503] Master 127.12.15.254:35951 was elected leader, sending a full tablet report...
I20260806 04:38:30.464121 7662 ts_tablet_manager.cc:1428] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 04:38:30.464519 7664 raft_consensus.cc:348] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.464777 7664 raft_consensus.cc:374] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:30.464843 7664 raft_consensus.cc:729] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37be2c449cee49eeaefe4e51a6c2780b, State: Initialized, Role: FOLLOWER
I20260806 04:38:30.465425 7664 consensus_queue.cc:260] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.466069 7238 heartbeater.cc:503] Master 127.12.15.254:35951 was elected leader, sending a full tablet report...
I20260806 04:38:30.466460 7664 ts_tablet_manager.cc:1428] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 04:38:30.504732 7240 tablet.cc:2369] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:30.533895 7671 raft_consensus.cc:482] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:30.534089 7671 raft_consensus.cc:504] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.535179 7671 leader_election.cc:288] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37be2c449cee49eeaefe4e51a6c2780b (127.12.15.193:34347), 5496e7c1cf7c46d58b3560af28d918ec (127.12.15.194:33517)
I20260806 04:38:30.538625 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.535417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:34347 (local address 127.12.15.195:59853)
0806 04:38:30.535545 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.535561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.535652 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.535920 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.535923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.535943 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:30.536053 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.536059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.536806 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.536809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.537567 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.537576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.538227 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.538231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.538327 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.538348 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.538414 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.538483 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 04:38:30.538625 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.535521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:59853 (local address 127.12.15.193:34347)
0806 04:38:30.535676 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.535681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.535702 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:38:30.535759 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.535761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.535816 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.535888 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:30.536162 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.536703 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.537698 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.538105 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.538233 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.538290 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.538437 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.538507 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 04:38:30.538923 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.535428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:33517 (local address 127.12.15.195:33817)
0806 04:38:30.535934 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.535944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:30.536036 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:30.536259 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:30.536262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:30.536277 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:30.536416 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.536422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.537139 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.537142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.537888 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:30.537896 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.538507 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.538510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:30.538606 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.538645 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 04:38:30.538710 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:38:30.538783 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":147,"thread_start_us":141,"threads_started":1}
I20260806 04:38:30.539041 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.535550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:33817 (local address 127.12.15.194:33517)
0806 04:38:30.535687 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.535692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.536019 (+ 327us) server_negotiation.cc:407] Connection header received
0806 04:38:30.536133 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.536136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.536182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.536243 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:30.536513 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.537033 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.538001 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.538398 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.538494 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.538540 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.538857 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.538934 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 04:38:30.539299 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3a7e0e37ad4be9b8516c150d74c4df" candidate_uuid: "cdbcea98d648450a8750e67fe19acf07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37be2c449cee49eeaefe4e51a6c2780b" is_pre_election: true
I20260806 04:38:30.539582 7162 raft_consensus.cc:2454] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdbcea98d648450a8750e67fe19acf07 in term 0.
I20260806 04:38:30.539583 7370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3a7e0e37ad4be9b8516c150d74c4df" candidate_uuid: "cdbcea98d648450a8750e67fe19acf07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5496e7c1cf7c46d58b3560af28d918ec" is_pre_election: true
I20260806 04:38:30.539837 7370 raft_consensus.cc:2454] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdbcea98d648450a8750e67fe19acf07 in term 0.
I20260806 04:38:30.540076 7467 leader_election.cc:302] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37be2c449cee49eeaefe4e51a6c2780b, cdbcea98d648450a8750e67fe19acf07; no voters:
I20260806 04:38:30.540364 7671 raft_consensus.cc:2781] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:38:30.540450 7671 raft_consensus.cc:482] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:38:30.540498 7671 raft_consensus.cc:3037] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:30.541486 7671 raft_consensus.cc:504] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.541797 7671 leader_election.cc:288] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [CANDIDATE]: Term 1 election: Requested vote from peers 37be2c449cee49eeaefe4e51a6c2780b (127.12.15.193:34347), 5496e7c1cf7c46d58b3560af28d918ec (127.12.15.194:33517)
I20260806 04:38:30.542109 7370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3a7e0e37ad4be9b8516c150d74c4df" candidate_uuid: "cdbcea98d648450a8750e67fe19acf07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5496e7c1cf7c46d58b3560af28d918ec"
I20260806 04:38:30.542145 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae3a7e0e37ad4be9b8516c150d74c4df" candidate_uuid: "cdbcea98d648450a8750e67fe19acf07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37be2c449cee49eeaefe4e51a6c2780b"
I20260806 04:38:30.542219 7370 raft_consensus.cc:3037] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:30.542243 7162 raft_consensus.cc:3037] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:30.543215 7162 raft_consensus.cc:2454] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdbcea98d648450a8750e67fe19acf07 in term 1.
I20260806 04:38:30.543228 7370 raft_consensus.cc:2454] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdbcea98d648450a8750e67fe19acf07 in term 1.
I20260806 04:38:30.543516 7469 leader_election.cc:302] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 37be2c449cee49eeaefe4e51a6c2780b, 5496e7c1cf7c46d58b3560af28d918ec, cdbcea98d648450a8750e67fe19acf07; no voters:
I20260806 04:38:30.543687 7671 raft_consensus.cc:2781] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:30.543889 7671 raft_consensus.cc:686] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 1 LEADER]: Becoming Leader. State: Replica: cdbcea98d648450a8750e67fe19acf07, State: Running, Role: LEADER
I20260806 04:38:30.544232 7671 consensus_queue.cc:237] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } }
I20260806 04:38:30.547344 6918 catalog_manager.cc:5495] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdbcea98d648450a8750e67fe19acf07 (127.12.15.195). New cstate: current_term: 1 leader_uuid: "cdbcea98d648450a8750e67fe19acf07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbcea98d648450a8750e67fe19acf07" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 44497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 } health_report { overall_health: UNKNOWN } } }
I20260806 04:38:30.587172 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.584037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.12.15.195:44497)
0806 04:38:30.584167 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:30.584170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:30.584184 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:38:30.584284 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:30.584286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:30.584359 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:30.584445 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:38:30.584632 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.585124 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.585912 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:30.586115 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:30.586181 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:30.586395 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:38:30.586813 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:38:30.586900 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:30.586953 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 04:38:30.586998 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:38:30.592881 7162 raft_consensus.cc:1261] T ae3a7e0e37ad4be9b8516c150d74c4df P 37be2c449cee49eeaefe4e51a6c2780b [term 1 FOLLOWER]: Refusing update from remote peer cdbcea98d648450a8750e67fe19acf07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:30.592882 7370 raft_consensus.cc:1261] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 1 FOLLOWER]: Refusing update from remote peer cdbcea98d648450a8750e67fe19acf07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:30.593524 7671 consensus_queue.cc:1035] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37be2c449cee49eeaefe4e51a6c2780b" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 34347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:30.593709 7675 consensus_queue.cc:1035] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5496e7c1cf7c46d58b3560af28d918ec" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:30.600827 7685 mvcc.cc:205] Tried to move back new op lower bound from 7315419588981530624 to 7315419588791803904. Current Snapshot: MvccSnapshot[applied={T|T < 7315419588981530624}]
I20260806 04:38:30.609542 6918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:30.609705 6918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:30.612084 6918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1281b5ef5110467a8f500bf784a2be5f: Sending DeleteTablet for 3 replicas of tablet ae3a7e0e37ad4be9b8516c150d74c4df
I20260806 04:38:30.612782 7339 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3a7e0e37ad4be9b8516c150d74c4df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:30 UTC) from {username='slave'} at 127.0.0.1:39330
I20260806 04:38:30.612788 7131 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3a7e0e37ad4be9b8516c150d74c4df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:30 UTC) from {username='slave'} at 127.0.0.1:50720
I20260806 04:38:30.612798 7543 tablet_service.cc:1514] Processing DeleteTablet for tablet ae3a7e0e37ad4be9b8516c150d74c4df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:30 UTC) from {username='slave'} at 127.0.0.1:36082
I20260806 04:38:30.613317 7702 tablet_replica.cc:330] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: stopping tablet replica
I20260806 04:38:30.613590 7702 raft_consensus.cc:2229] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:30.613893 7702 raft_consensus.cc:2258] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:30.613916 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 7020
I20260806 04:38:30.616492 7699 tablet_replica.cc:330] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: stopping tablet replica
I20260806 04:38:30.616745 7699 raft_consensus.cc:2229] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:38:30.617058 7699 raft_consensus.cc:2258] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:30.621152 7699 ts_tablet_manager.cc:1905] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:30.623665 7702 ts_tablet_manager.cc:1905] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:30.623886 7699 ts_tablet_manager.cc:1918] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:30.623982 7699 log.cc:1196] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/ae3a7e0e37ad4be9b8516c150d74c4df
I20260806 04:38:30.624297 7699 ts_tablet_manager.cc:1939] T ae3a7e0e37ad4be9b8516c150d74c4df P cdbcea98d648450a8750e67fe19acf07: Deleting consensus metadata
I20260806 04:38:30.626271 7702 ts_tablet_manager.cc:1918] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:30.626358 7702 log.cc:1196] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/ae3a7e0e37ad4be9b8516c150d74c4df
I20260806 04:38:30.626606 7702 ts_tablet_manager.cc:1939] T ae3a7e0e37ad4be9b8516c150d74c4df P 5496e7c1cf7c46d58b3560af28d918ec: Deleting consensus metadata
I20260806 04:38:30.627599 6901 catalog_manager.cc:4841] TS 5496e7c1cf7c46d58b3560af28d918ec (127.12.15.194:33517): tablet ae3a7e0e37ad4be9b8516c150d74c4df (table test-table [id=0eddde2a799e4ed58f6cfbab0c40053c]) successfully deleted
I20260806 04:38:30.647367 6900 catalog_manager.cc:4841] TS cdbcea98d648450a8750e67fe19acf07 (127.12.15.195:44497): tablet ae3a7e0e37ad4be9b8516c150d74c4df (table test-table [id=0eddde2a799e4ed58f6cfbab0c40053c]) successfully deleted
I20260806 04:38:30.653162 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 7243
W20260806 04:38:30.653120 6899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:34347 (error 108)
I20260806 04:38:30.656029 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:30.655658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:34347 (local address 127.0.0.1:50736)
0806 04:38:30.655860 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:30.655934 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:34347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260806 04:38:30.656204 6899 catalog_manager.cc:4584] TS 37be2c449cee49eeaefe4e51a6c2780b (127.12.15.193:34347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae3a7e0e37ad4be9b8516c150d74c4df: Network error: Client connection negotiation failed: client connection to 127.12.15.193:34347: connect: Connection refused (error 111)
I20260806 04:38:30.661687 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 7444
I20260806 04:38:30.675925 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 6876
2026-08-06T04:38:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1185 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:30 dist-test-slave-xt4k krb5kdc[7709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:30 dist-test-slave-xt4k krb5kdc[7709](info): set up 2 sockets
Aug 06 04:38:30 dist-test-slave-xt4k krb5kdc[7709](info): commencing operation
krb5kdc: starting...
W20260806 04:38:31.817013 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:31 dist-test-slave-xt4k krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:38:31.943408 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40167
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:40167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:32.056469 7725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:32.056723 7725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:32.056777 7725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:32.060755 7725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:32.060807 7725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:32.060832 7725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:32.060854 7725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:32.060883 7725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:32.066121 7725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:40167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:40167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:32.067139 7725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:32.068159 7725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:32.076453 7734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.076462 7737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.076464 7733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:32.076552 7725 server_base.cc:1034] running on GCE node
I20260806 04:38:32.077304 7725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:32.078373 7725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:32.079533 7725 hybrid_clock.cc:648] HybridClock initialized: now 1785991112079515 us; error 30 us; skew 500 ppm
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:32.086738 7725 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:38:32.087641 7725 webserver.cc:460] Webserver started at http://127.12.15.254:40329/ using document root <none> and password file <none>
I20260806 04:38:32.088196 7725 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:32.088264 7725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:32.088515 7725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:32.090613 7725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "6b3be6cae491486686d4556bc593a97c"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "87b1f4ebf335c2beaf68fff5c458c071"
server_key_iv: "16a3e8b38f3a5976dca484b9288a301b"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.091172 7725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "6b3be6cae491486686d4556bc593a97c"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "87b1f4ebf335c2beaf68fff5c458c071"
server_key_iv: "16a3e8b38f3a5976dca484b9288a301b"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.094866 7725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 04:38:32.097240 7748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.098511 7725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:32.098713 7725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "6b3be6cae491486686d4556bc593a97c"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "87b1f4ebf335c2beaf68fff5c458c071"
server_key_iv: "16a3e8b38f3a5976dca484b9288a301b"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.098835 7725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:32.116632 7725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:32.119534 7725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:32.119727 7725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:32.126711 7725 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:40167
I20260806 04:38:32.126727 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:40167 every 8 connection(s)
I20260806 04:38:32.127703 7725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:32.130539 7838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:32.130824 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 7725
I20260806 04:38:32.130972 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:32.131291 12351 external_mini_cluster.cc:1428] Setting key ad9bdec1d91fe8948542d5dfee72ea5b
I20260806 04:38:32.137583 7838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c: Bootstrap starting.
I20260806 04:38:32.140159 7838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c: Neither blocks nor log segments found. Creating new log.
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:32.141073 7838 log.cc:824] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:32.143138 7838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c: No bootstrap required, opened a new log
I20260806 04:38:32.145525 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.132649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.12.15.254:40167)
0806 04:38:32.133063 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.133082 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.133124 (+ 42us) server_negotiation.cc:407] Connection header received
0806 04:38:32.133908 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.133944 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.134353 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.134706 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:32.135349 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.136840 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.137628 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.138279 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.138439 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.141149 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:32.141155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:32.141168 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:32.141195 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:32.142866 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.143333 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.143338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.143343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.143455 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.143729 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.143731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.143732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.144107 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:32.144241 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.144551 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.144821 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260806 04:38:32.147081 7838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3be6cae491486686d4556bc593a97c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40167 } }
I20260806 04:38:32.147336 7838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:32.147404 7838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3be6cae491486686d4556bc593a97c, State: Initialized, Role: FOLLOWER
I20260806 04:38:32.147795 7838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3be6cae491486686d4556bc593a97c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40167 } }
I20260806 04:38:32.147915 7838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:32.147961 7838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:32.148028 7838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:32.148866 7838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3be6cae491486686d4556bc593a97c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40167 } }
I20260806 04:38:32.149171 7838 leader_election.cc:302] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b3be6cae491486686d4556bc593a97c; no voters:
I20260806 04:38:32.149504 7838 leader_election.cc:288] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:32.149581 7843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:32.149765 7843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [term 1 LEADER]: Becoming Leader. State: Replica: 6b3be6cae491486686d4556bc593a97c, State: Running, Role: LEADER
I20260806 04:38:32.150064 7843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3be6cae491486686d4556bc593a97c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40167 } }
I20260806 04:38:32.150763 7838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:32.151717 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b3be6cae491486686d4556bc593a97c. Latest consensus state: current_term: 1 leader_uuid: "6b3be6cae491486686d4556bc593a97c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3be6cae491486686d4556bc593a97c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40167 } } }
I20260806 04:38:32.151849 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:32.152163 7846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b3be6cae491486686d4556bc593a97c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3be6cae491486686d4556bc593a97c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 40167 } } }
I20260806 04:38:32.152230 7858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:32.152287 7846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:32.155485 7858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:32.160645 7858 catalog_manager.cc:1347] Generated new cluster ID: 6cf89f026e4645569d110a43447aa2d9
I20260806 04:38:32.160718 7858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:32.170501 7858 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:32.171485 7858 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:32.178639 7858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c: Generated new TSK 0
I20260806 04:38:32.179396 7858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:38:32.219444 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40167
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:32.335340 7883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:32.335615 7883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:32.335675 7883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:32.339633 7883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:32.339686 7883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:32.339744 7883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:32.344947 7883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:32.345937 7883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:32.346885 7883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:32.356221 7893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.356221 7890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.356238 7888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:32.356415 7883 server_base.cc:1034] running on GCE node
I20260806 04:38:32.356876 7883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:32.357510 7883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:32.358685 7883 hybrid_clock.cc:648] HybridClock initialized: now 1785991112358665 us; error 28 us; skew 500 ppm
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:32.361258 7883 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:38:32.362082 7883 webserver.cc:460] Webserver started at http://127.12.15.193:41221/ using document root <none> and password file <none>
I20260806 04:38:32.362632 7883 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:32.362700 7883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:32.362887 7883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:32.364970 7883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "48cc3cfc70c740049b6ec00c34796245"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "13f0a934006a03c4abbd329dd2d49162"
server_key_iv: "7a5227a1b98c80cb0bfd6f698e9e124b"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.365491 7883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "48cc3cfc70c740049b6ec00c34796245"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "13f0a934006a03c4abbd329dd2d49162"
server_key_iv: "7a5227a1b98c80cb0bfd6f698e9e124b"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.369036 7883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:38:32.371323 7904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.372603 7883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:32.372722 7883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "48cc3cfc70c740049b6ec00c34796245"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "13f0a934006a03c4abbd329dd2d49162"
server_key_iv: "7a5227a1b98c80cb0bfd6f698e9e124b"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.372812 7883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:32.392326 7883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:32.395200 7883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:32.395390 7883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:32.396008 7883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:32.396943 7883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:32.396997 7883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.397053 7883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:32.397087 7883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.405655 7883 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:40201
I20260806 04:38:32.405699 8079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:40201 every 8 connection(s)
I20260806 04:38:32.406518 7883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:32.406836 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 7883
I20260806 04:38:32.406988 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:38:32.407321 12351 external_mini_cluster.cc:1428] Setting key 39da831e2a4029ee819718b7f8febb48
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:38:32.420758 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.408207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:51499 (local address 127.12.15.254:40167)
0806 04:38:32.408329 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.408332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.409395 (+ 1063us) server_negotiation.cc:407] Connection header received
0806 04:38:32.410751 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.410754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.410799 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.410867 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:32.412322 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.412886 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.414525 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.414777 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.414866 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.417552 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:32.417555 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:32.417557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:32.417581 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:32.418846 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.419306 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.419309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.419310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.419381 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.419658 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.419660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.419661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.419860 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:32.419959 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.420443 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.420577 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 04:38:32.421520 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.408757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40167 (local address 127.12.15.193:51499)
0806 04:38:32.409240 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:32.409292 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:32.410514 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:32.411042 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:32.411051 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:32.411501 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:32.412162 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.412181 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.413016 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.413020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.414387 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.414400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.414936 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.414941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.415138 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.415757 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:32.415776 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:32.417391 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:32.418948 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:32.418953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:32.418964 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:32.419229 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:32.419447 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:32.419449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:32.419451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:32.419577 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:32.419940 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:32.419945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:32.420181 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 04:38:32.420385 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 04:38:32.420707 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":113,"threads_started":1}
I20260806 04:38:32.422788 8080 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40167
I20260806 04:38:32.423015 8080 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:32.423539 8080 heartbeater.cc:511] Master 127.12.15.254:40167 requested a full tablet report, sending...
I20260806 04:38:32.425362 7776 ts_manager.cc:194] Registered new tserver with Master: 48cc3cfc70c740049b6ec00c34796245 (127.12.15.193:40201)
I20260806 04:38:32.426939 7776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:51499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:38:32.437147 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40167
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:32.547555 8088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:32.547828 8088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:32.547885 8088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:32.552199 8088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:32.552280 8088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:32.552381 8088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:32.558090 8088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:32.559228 8088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:32.560267 8088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:32.570495 8094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.570518 8099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:32.570575 8088 server_base.cc:1034] running on GCE node
W20260806 04:38:32.570495 8096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:32.571151 8088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:32.571791 8088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:32.572942 8088 hybrid_clock.cc:648] HybridClock initialized: now 1785991112572922 us; error 33 us; skew 500 ppm
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:32.575752 8088 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:38:32.576750 8088 webserver.cc:460] Webserver started at http://127.12.15.194:46325/ using document root <none> and password file <none>
I20260806 04:38:32.577327 8088 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:32.577406 8088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:32.577626 8088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:32.579711 8088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "609990368288437b84d464df3abff503"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "dd6b0b00bb9da6e9e5fc977d58fa403d"
server_key_iv: "61f03e79dc1f9f85c07bad37462e4517"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.580262 8088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "609990368288437b84d464df3abff503"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "dd6b0b00bb9da6e9e5fc977d58fa403d"
server_key_iv: "61f03e79dc1f9f85c07bad37462e4517"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.584092 8088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 04:38:32.586585 8109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.587889 8088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 04:38:32.588028 8088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "609990368288437b84d464df3abff503"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "dd6b0b00bb9da6e9e5fc977d58fa403d"
server_key_iv: "61f03e79dc1f9f85c07bad37462e4517"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.588129 8088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:32.617112 8088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:32.620493 8088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:32.620682 8088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:32.621294 8088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:32.622140 8088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:32.622188 8088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.622242 8088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:32.622272 8088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.630604 8088 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:45011
I20260806 04:38:32.630640 8286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:45011 every 8 connection(s)
I20260806 04:38:32.631491 8088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:32.634460 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 8088
I20260806 04:38:32.634605 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:38:32.634949 12351 external_mini_cluster.cc:1428] Setting key f741212a91b78cc3cfd6bd5772d06a17
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:38:32.645900 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.633354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:37289 (local address 127.12.15.254:40167)
0806 04:38:32.633507 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.633511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.634347 (+ 836us) server_negotiation.cc:407] Connection header received
0806 04:38:32.635932 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.635935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.635979 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.636039 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:32.637552 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.638132 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.639403 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.639623 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.639716 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.642329 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:32.642333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:32.642335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:32.642361 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:32.643712 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.644207 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.644210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.644211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.644287 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.644633 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.644636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.644638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.644833 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:32.644931 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.645548 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.645695 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 04:38:32.646620 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.633702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40167 (local address 127.12.15.194:37289)
0806 04:38:32.634168 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:32.634225 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:32.635716 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:32.636236 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:32.636243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:32.636727 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:32.637387 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.637408 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.638255 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.638259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.639283 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.639293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.639768 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.639773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.639957 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.640621 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:32.640640 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:32.642153 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:32.643833 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:32.643839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:32.643855 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:32.644111 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:32.644393 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:32.644396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:32.644398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:32.644543 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:32.644960 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:32.644968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:32.645253 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 04:38:32.645412 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 04:38:32.645677 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:38:32.647801 8288 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40167
I20260806 04:38:32.648033 8288 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:32.648579 8288 heartbeater.cc:511] Master 127.12.15.254:40167 requested a full tablet report, sending...
I20260806 04:38:32.649623 7776 ts_manager.cc:194] Registered new tserver with Master: 609990368288437b84d464df3abff503 (127.12.15.194:45011)
I20260806 04:38:32.650177 7776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:37289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:38:32.663059 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40167
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:32.772488 8295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:32.772715 8295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:32.772755 8295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:32.776626 8295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:32.776674 8295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:32.776727 8295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:32.781759 8295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:40167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:32.782691 8295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:32.783623 8295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:38:32.793078 8295 server_base.cc:1034] running on GCE node
W20260806 04:38:32.793004 8302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.792989 8305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:32.793015 8301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:32.793625 8295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:32.794250 8295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:32.795418 8295 hybrid_clock.cc:648] HybridClock initialized: now 1785991112795391 us; error 40 us; skew 500 ppm
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:32.798189 8295 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:38:32.799223 8295 webserver.cc:460] Webserver started at http://127.12.15.195:46717/ using document root <none> and password file <none>
I20260806 04:38:32.799896 8295 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:32.799979 8295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:32.800215 8295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:32.803000 8295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "948a570641214ac4a6ad5157d21be558"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "33d577a6c192bfb38bf49b49bdcda1fe"
server_key_iv: "0cb9496d84b2f0928796fb07d458506a"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.803699 8295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "948a570641214ac4a6ad5157d21be558"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "33d577a6c192bfb38bf49b49bdcda1fe"
server_key_iv: "0cb9496d84b2f0928796fb07d458506a"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.807667 8295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 04:38:32.810104 8316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.811297 8295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:32.811394 8295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "948a570641214ac4a6ad5157d21be558"
format_stamp: "Formatted at 2026-08-06 04:38:32 on dist-test-slave-xt4k"
server_key: "33d577a6c192bfb38bf49b49bdcda1fe"
server_key_iv: "0cb9496d84b2f0928796fb07d458506a"
server_key_version: "encryptionkey@0"
I20260806 04:38:32.811475 8295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:32.829193 8295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:32.832094 8295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:32.832243 8295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:32.832809 8295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:32.833595 8295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:32.833639 8295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.833678 8295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:32.833698 8295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:32.841867 8295 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:38243
I20260806 04:38:32.841902 8493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:38243 every 8 connection(s)
I20260806 04:38:32.842753 8295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:38:32.849638 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 8295
I20260806 04:38:32.849769 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:38:32.850033 12351 external_mini_cluster.cc:1428] Setting key 19ff5d8cebb89599a1deb16397e78bd4
Aug 06 04:38:32 dist-test-slave-xt4k krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991112, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:32.855808 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.844514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:44837 (local address 127.12.15.254:40167)
0806 04:38:32.844659 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.844663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.845427 (+ 764us) server_negotiation.cc:407] Connection header received
0806 04:38:32.846766 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.846770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.846826 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.846899 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:32.848024 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.848614 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.849717 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.849978 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.850073 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.852840 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:32.852844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:32.852846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:32.852873 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:32.853927 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.854390 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.854394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.854396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.854475 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.854783 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.854787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.854789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.854966 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:32.855050 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.855523 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.855650 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:38:32.856587 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.844858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:40167 (local address 127.12.15.195:44837)
0806 04:38:32.845284 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:32.845325 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:32.846566 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:32.847085 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:32.847091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:32.847426 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:32.847885 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.847898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.848732 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.848735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.849601 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.849611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.850121 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.850125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.850308 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.850824 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:32.850841 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:32.852688 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:32.854031 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:32.854036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:32.854050 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:32.854298 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:32.854558 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:32.854560 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:32.854562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:32.854696 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:32.855062 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:32.855069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:32.855306 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 04:38:32.855462 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 04:38:32.855767 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260806 04:38:32.857561 8494 heartbeater.cc:347] Connected to a master server at 127.12.15.254:40167
I20260806 04:38:32.857793 8494 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:32.858254 8494 heartbeater.cc:511] Master 127.12.15.254:40167 requested a full tablet report, sending...
I20260806 04:38:32.859197 7776 ts_manager.cc:194] Registered new tserver with Master: 948a570641214ac4a6ad5157d21be558 (127.12.15.195:38243)
I20260806 04:38:32.859745 7776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:44837
I20260806 04:38:32.863718 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:38:32.872709 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.865727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.12.15.254:40167)
0806 04:38:32.865875 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.865879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.866092 (+ 213us) server_negotiation.cc:407] Connection header received
0806 04:38:32.866135 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.866137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.866178 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.866246 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:32.866899 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.867418 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.868172 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.868425 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.868516 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.869866 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:32.869869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:32.869871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:32.869895 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:32.871194 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.871630 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.871632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.871634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.871705 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:32.871930 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:32.871932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:32.871933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:32.872123 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:32.872231 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.872496 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.872573 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 04:38:32.875572 7776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:38:32.878814 7776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:38:32.893271 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.887434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:38243 (local address 127.0.0.1:38862)
0806 04:38:32.887762 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:32.887801 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:32.887941 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:32.888668 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:32.888674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:32.888700 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:32.888865 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.888874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.890123 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.890126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.891757 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.891768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.892711 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.892715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.892843 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.892889 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 04:38:32.892991 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 04:38:32.893121 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260806 04:38:32.893280 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.887625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.12.15.195:38243)
0806 04:38:32.888018 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.888030 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.888055 (+ 25us) server_negotiation.cc:407] Connection header received
0806 04:38:32.888134 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.888140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.888400 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.888562 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:32.889125 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.890015 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.891885 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.892594 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.892699 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.892986 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.893040 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.893135 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":161,"threads_started":1}
I20260806 04:38:32.893501 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.888192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:45011 (local address 127.0.0.1:36004)
0806 04:38:32.888763 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:32.888777 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:32.888872 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:32.889753 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:32.889757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:32.889777 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:32.889889 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.889895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.890879 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.890883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.891948 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.891960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.893100 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.893104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.893205 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.893232 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 04:38:32.893279 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 04:38:32.893344 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":95,"threads_started":1}
I20260806 04:38:32.894106 8514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.888388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.12.15.194:45011)
0806 04:38:32.889312 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.889324 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.889347 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:38:32.889426 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.889430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.889605 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.889756 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:32.890010 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.890751 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.892082 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.892964 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.893693 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.893838 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.893888 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.893976 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":99,"threads_started":1}
I20260806 04:38:32.894361 8515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.888593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:40201 (local address 127.0.0.1:50556)
0806 04:38:32.889026 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:32.889036 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:32.889099 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:32.889916 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:32.889919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:32.889936 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:32.890041 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.890049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.891426 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.891429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.892582 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:32.892594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.893978 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.893981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:32.894081 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.894103 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:38:32.894168 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:38:32.894237 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":62,"thread_start_us":82,"threads_started":1}
I20260806 04:38:32.894630 8516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:32.888997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.12.15.193:40201)
0806 04:38:32.889312 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:32.889326 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:32.889347 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:38:32.889419 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:32.889424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:32.889605 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:32.889756 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:32.890178 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.891256 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.892749 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:32.893859 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:32.893967 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:32.894102 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:32.894334 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 04:38:32.894498 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":107,"threads_started":1}
I20260806 04:38:32.896903 7981 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0df06ade704731a404f1fa7cf24e54 (DEFAULT_TABLE table=test-table [id=2607fc2c07034f6298137f80a1ddba6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:32.896823 8391 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0df06ade704731a404f1fa7cf24e54 (DEFAULT_TABLE table=test-table [id=2607fc2c07034f6298137f80a1ddba6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:32.896824 8188 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0df06ade704731a404f1fa7cf24e54 (DEFAULT_TABLE table=test-table [id=2607fc2c07034f6298137f80a1ddba6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:32.897861 7981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0df06ade704731a404f1fa7cf24e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:32.897881 8188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0df06ade704731a404f1fa7cf24e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:32.897881 8391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0df06ade704731a404f1fa7cf24e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:32.904664 8517 tablet_bootstrap.cc:492] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: Bootstrap starting.
I20260806 04:38:32.905607 8518 tablet_bootstrap.cc:492] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Bootstrap starting.
I20260806 04:38:32.905607 8519 tablet_bootstrap.cc:492] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Bootstrap starting.
I20260806 04:38:32.907049 8517 tablet_bootstrap.cc:654] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:32.907958 8519 tablet_bootstrap.cc:654] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:32.907958 8518 tablet_bootstrap.cc:654] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:32.908272 8517 log.cc:824] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:32.909327 8518 log.cc:824] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:32.909327 8519 log.cc:824] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:32.910367 8517 tablet_bootstrap.cc:492] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: No bootstrap required, opened a new log
I20260806 04:38:32.910578 8517 ts_tablet_manager.cc:1397] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 04:38:32.911338 8519 tablet_bootstrap.cc:492] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: No bootstrap required, opened a new log
I20260806 04:38:32.911535 8519 ts_tablet_manager.cc:1397] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 04:38:32.911631 8518 tablet_bootstrap.cc:492] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: No bootstrap required, opened a new log
I20260806 04:38:32.911818 8518 ts_tablet_manager.cc:1397] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:38:32.913704 8517 raft_consensus.cc:348] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:32.913923 8517 raft_consensus.cc:374] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:32.913975 8517 raft_consensus.cc:729] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48cc3cfc70c740049b6ec00c34796245, State: Initialized, Role: FOLLOWER
I20260806 04:38:32.914433 8517 consensus_queue.cc:260] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:32.915107 8080 heartbeater.cc:503] Master 127.12.15.254:40167 was elected leader, sending a full tablet report...
I20260806 04:38:32.915442 8517 ts_tablet_manager.cc:1428] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260806 04:38:32.915405 8519 raft_consensus.cc:348] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:32.915405 8518 raft_consensus.cc:348] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:32.915645 8519 raft_consensus.cc:374] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:32.915655 8518 raft_consensus.cc:374] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:32.915706 8518 raft_consensus.cc:729] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948a570641214ac4a6ad5157d21be558, State: Initialized, Role: FOLLOWER
I20260806 04:38:32.915707 8519 raft_consensus.cc:729] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609990368288437b84d464df3abff503, State: Initialized, Role: FOLLOWER
I20260806 04:38:32.916157 8518 consensus_queue.cc:260] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:32.916162 8519 consensus_queue.cc:260] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:32.917136 8518 ts_tablet_manager.cc:1428] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:38:32.917136 8519 ts_tablet_manager.cc:1428] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 04:38:32.917482 8288 heartbeater.cc:503] Master 127.12.15.254:40167 was elected leader, sending a full tablet report...
I20260806 04:38:32.917505 8494 heartbeater.cc:503] Master 127.12.15.254:40167 was elected leader, sending a full tablet report...
W20260806 04:38:33.093712 8495 tablet.cc:2369] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:38:33.132587 8291 tablet.cc:2369] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:38:33.157893 8081 tablet.cc:2369] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:33.290916 8532 raft_consensus.cc:482] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:33.291195 8532 raft_consensus.cc:504] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:33.292488 8532 leader_election.cc:288] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948a570641214ac4a6ad5157d21be558 (127.12.15.195:38243), 48cc3cfc70c740049b6ec00c34796245 (127.12.15.193:40201)
I20260806 04:38:33.296267 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.292857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:34497 (local address 127.12.15.195:38243)
0806 04:38:33.292997 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:33.293001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:33.293110 (+ 109us) server_negotiation.cc:407] Connection header received
0806 04:38:33.293329 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:33.293333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:33.293392 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:33.293488 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:33.293785 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.294301 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.295370 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.295765 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.295870 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.295924 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:33.296083 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 04:38:33.296156 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 04:38:33.296290 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.292760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:38243 (local address 127.12.15.194:34497)
0806 04:38:33.293019 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:33.293034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:33.293158 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:33.293503 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:33.293506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:33.293529 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:33.293678 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:33.293684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.294413 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.294416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:33.295254 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:33.295263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.295874 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.295878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:33.295972 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.295995 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:38:33.296062 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 04:38:33.296131 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":20,"spinlock_wait_cycles":7040,"thread_start_us":84,"threads_started":1}
I20260806 04:38:33.296474 8516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.293290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:35559 (local address 127.12.15.193:40201)
0806 04:38:33.293413 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:33.293417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:33.293550 (+ 133us) server_negotiation.cc:407] Connection header received
0806 04:38:33.293659 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:33.293662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:33.293708 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:33.293771 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:33.294028 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.294504 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.295538 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.295924 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.296027 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.296077 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:33.296238 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 04:38:33.296309 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:38:33.296604 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.293158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:40201 (local address 127.12.15.194:35559)
0806 04:38:33.293449 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:33.293460 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:33.293571 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:33.293788 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:33.293791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:33.293806 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:33.293918 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:33.293924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.294605 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.294609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:33.295414 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:33.295423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.296028 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.296032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:33.296126 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.296148 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:38:33.296209 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 04:38:33.296278 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260806 04:38:33.297255 8008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0df06ade704731a404f1fa7cf24e54" candidate_uuid: "609990368288437b84d464df3abff503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48cc3cfc70c740049b6ec00c34796245" is_pre_election: true
I20260806 04:38:33.297260 8424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0df06ade704731a404f1fa7cf24e54" candidate_uuid: "609990368288437b84d464df3abff503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948a570641214ac4a6ad5157d21be558" is_pre_election: true
I20260806 04:38:33.297539 8008 raft_consensus.cc:2454] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609990368288437b84d464df3abff503 in term 0.
I20260806 04:38:33.297546 8424 raft_consensus.cc:2454] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609990368288437b84d464df3abff503 in term 0.
I20260806 04:38:33.297976 8115 leader_election.cc:302] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48cc3cfc70c740049b6ec00c34796245, 609990368288437b84d464df3abff503; no voters:
I20260806 04:38:33.298226 8532 raft_consensus.cc:2781] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:38:33.298303 8532 raft_consensus.cc:482] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:38:33.298334 8532 raft_consensus.cc:3037] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:33.299366 8532 raft_consensus.cc:504] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:33.299693 8532 leader_election.cc:288] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [CANDIDATE]: Term 1 election: Requested vote from peers 948a570641214ac4a6ad5157d21be558 (127.12.15.195:38243), 48cc3cfc70c740049b6ec00c34796245 (127.12.15.193:40201)
I20260806 04:38:33.300051 8424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0df06ade704731a404f1fa7cf24e54" candidate_uuid: "609990368288437b84d464df3abff503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948a570641214ac4a6ad5157d21be558"
I20260806 04:38:33.300076 8008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0df06ade704731a404f1fa7cf24e54" candidate_uuid: "609990368288437b84d464df3abff503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48cc3cfc70c740049b6ec00c34796245"
I20260806 04:38:33.300155 8424 raft_consensus.cc:3037] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:33.300179 8008 raft_consensus.cc:3037] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:33.301182 8424 raft_consensus.cc:2454] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609990368288437b84d464df3abff503 in term 1.
I20260806 04:38:33.301189 8008 raft_consensus.cc:2454] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609990368288437b84d464df3abff503 in term 1.
I20260806 04:38:33.301457 8115 leader_election.cc:302] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609990368288437b84d464df3abff503, 948a570641214ac4a6ad5157d21be558; no voters:
I20260806 04:38:33.301620 8532 raft_consensus.cc:2781] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:33.301818 8532 raft_consensus.cc:686] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 1 LEADER]: Becoming Leader. State: Replica: 609990368288437b84d464df3abff503, State: Running, Role: LEADER
I20260806 04:38:33.302199 8532 consensus_queue.cc:237] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } }
I20260806 04:38:33.305692 7775 catalog_manager.cc:5495] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 reported cstate change: term changed from 0 to 1, leader changed from <none> to 609990368288437b84d464df3abff503 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "609990368288437b84d464df3abff503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "609990368288437b84d464df3abff503" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 } health_report { overall_health: UNKNOWN } } }
I20260806 04:38:33.352125 8514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.348769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.12.15.194:45011)
0806 04:38:33.348889 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:33.348893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:33.348908 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:38:33.349035 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:33.349037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:33.349096 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:33.349182 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:38:33.349378 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.349878 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.350713 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.350920 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.350995 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.351263 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:38:33.351732 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:38:33.351846 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:33.351939 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 04:38:33.351978 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 04:38:33.358116 8424 raft_consensus.cc:1261] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 1 FOLLOWER]: Refusing update from remote peer 609990368288437b84d464df3abff503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:33.358125 8008 raft_consensus.cc:1261] T 2b0df06ade704731a404f1fa7cf24e54 P 48cc3cfc70c740049b6ec00c34796245 [term 1 FOLLOWER]: Refusing update from remote peer 609990368288437b84d464df3abff503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:33.358834 8532 consensus_queue.cc:1035] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48cc3cfc70c740049b6ec00c34796245" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 40201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:33.359023 8535 consensus_queue.cc:1035] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948a570641214ac4a6ad5157d21be558" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 38243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:33.377982 7776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:33.378154 7776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:33.380667 7776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b3be6cae491486686d4556bc593a97c: Sending DeleteTablet for 3 replicas of tablet 2b0df06ade704731a404f1fa7cf24e54
I20260806 04:38:33.381426 8188 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0df06ade704731a404f1fa7cf24e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:33 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:36004
I20260806 04:38:33.381414 8391 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0df06ade704731a404f1fa7cf24e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:33 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:38862
I20260806 04:38:33.381500 7981 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0df06ade704731a404f1fa7cf24e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:33 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:50556
I20260806 04:38:33.381927 8555 tablet_replica.cc:330] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: stopping tablet replica
I20260806 04:38:33.382176 8558 tablet_replica.cc:330] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: stopping tablet replica
I20260806 04:38:33.382232 8555 raft_consensus.cc:2229] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:38:33.382537 8558 raft_consensus.cc:2229] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:33.382668 8555 raft_consensus.cc:2258] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:33.382817 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 7883
I20260806 04:38:33.383030 8558 raft_consensus.cc:2258] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:33.386718 8558 ts_tablet_manager.cc:1905] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:33.388063 8555 ts_tablet_manager.cc:1905] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:33.390592 8558 ts_tablet_manager.cc:1918] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:33.390798 8558 log.cc:1196] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/2b0df06ade704731a404f1fa7cf24e54
I20260806 04:38:33.391184 8558 ts_tablet_manager.cc:1939] T 2b0df06ade704731a404f1fa7cf24e54 P 948a570641214ac4a6ad5157d21be558: Deleting consensus metadata
I20260806 04:38:33.391301 8555 ts_tablet_manager.cc:1918] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:33.391431 8555 log.cc:1196] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/2b0df06ade704731a404f1fa7cf24e54
I20260806 04:38:33.391820 8555 ts_tablet_manager.cc:1939] T 2b0df06ade704731a404f1fa7cf24e54 P 609990368288437b84d464df3abff503: Deleting consensus metadata
I20260806 04:38:33.392901 7750 catalog_manager.cc:4841] TS 948a570641214ac4a6ad5157d21be558 (127.12.15.195:38243): tablet 2b0df06ade704731a404f1fa7cf24e54 (table test-table [id=2607fc2c07034f6298137f80a1ddba6f]) successfully deleted
I20260806 04:38:33.396322 7750 catalog_manager.cc:4841] TS 609990368288437b84d464df3abff503 (127.12.15.194:45011): tablet 2b0df06ade704731a404f1fa7cf24e54 (table test-table [id=2607fc2c07034f6298137f80a1ddba6f]) successfully deleted
W20260806 04:38:33.404829 7750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:40201 (error 108)
I20260806 04:38:33.404934 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 8088
I20260806 04:38:33.408818 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.408322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:40201 (local address 127.0.0.1:50558)
0806 04:38:33.408626 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:33.408705 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":75,"threads_started":1}
W20260806 04:38:33.409042 7750 catalog_manager.cc:4584] TS 48cc3cfc70c740049b6ec00c34796245 (127.12.15.193:40201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b0df06ade704731a404f1fa7cf24e54: Network error: Client connection negotiation failed: client connection to 127.12.15.193:40201: connect: Connection refused (error 111)
I20260806 04:38:33.414049 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 8295
I20260806 04:38:33.423012 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 7725
2026-08-06T04:38:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2748 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T04:38:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:33Z Disabled control of system clock
I20260806 04:38:33.459681 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36649
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:36649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 04:38:33.569463 8567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:33.569706 8567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:33.569739 8567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:33.573601 8567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:33.573647 8567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:33.573661 8567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:33.573676 8567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:33.573689 8567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:33.578356 8567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:36649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:36649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:33.579332 8567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:33.580660 8567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:33.588758 8581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:33.588753 8576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:33.588742 8577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:33.588835 8567 server_base.cc:1034] running on GCE node
I20260806 04:38:33.589529 8567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:33.590612 8567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:33.591796 8567 hybrid_clock.cc:648] HybridClock initialized: now 1785991113591779 us; error 27 us; skew 500 ppm
I20260806 04:38:33.593818 8567 webserver.cc:460] Webserver started at http://127.12.15.254:44563/ using document root <none> and password file <none>
I20260806 04:38:33.594374 8567 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:33.594437 8567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:33.594650 8567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:33.596745 8567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "ffb4300b24764681a185da5a4c6e6440"
format_stamp: "Formatted at 2026-08-06 04:38:33 on dist-test-slave-xt4k"
server_key: "9ecc52c2aa9d9e366054faffc21fc084"
server_key_iv: "1fbd05da312650a43e2ec5272300aa93"
server_key_version: "encryptionkey@0"
I20260806 04:38:33.597289 8567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "ffb4300b24764681a185da5a4c6e6440"
format_stamp: "Formatted at 2026-08-06 04:38:33 on dist-test-slave-xt4k"
server_key: "9ecc52c2aa9d9e366054faffc21fc084"
server_key_iv: "1fbd05da312650a43e2ec5272300aa93"
server_key_version: "encryptionkey@0"
I20260806 04:38:33.600872 8567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:38:33.603229 8589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:33.604578 8567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:33.604712 8567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "ffb4300b24764681a185da5a4c6e6440"
format_stamp: "Formatted at 2026-08-06 04:38:33 on dist-test-slave-xt4k"
server_key: "9ecc52c2aa9d9e366054faffc21fc084"
server_key_iv: "1fbd05da312650a43e2ec5272300aa93"
server_key_version: "encryptionkey@0"
I20260806 04:38:33.604808 8567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:33.623793 8567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:33.624532 8567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:33.624715 8567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:33.631603 8567 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:36649
I20260806 04:38:33.631623 8668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:36649 every 8 connection(s)
I20260806 04:38:33.632562 8567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:33.635643 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 8567
I20260806 04:38:33.635785 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:33.635785 8669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:33.636096 12351 external_mini_cluster.cc:1428] Setting key b4e678e880b7b41c4a7ed0d5e835eaae
I20260806 04:38:33.642486 8669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440: Bootstrap starting.
I20260806 04:38:33.644737 8669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:33.645475 8669 log.cc:824] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:33.645498 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.637616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57842 (local address 127.12.15.254:36649)
0806 04:38:33.638245 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:33.638260 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:33.638293 (+ 33us) server_negotiation.cc:407] Connection header received
0806 04:38:33.639196 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:33.639214 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:33.639712 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:33.640046 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:33.640497 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.641474 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.642328 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.642845 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.642994 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.643833 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:33.643838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:33.643850 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:33.644222 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:33.644275 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:33.644295 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:33.644421 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:33.644539 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 04:38:33.644794 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":141,"threads_started":1}
I20260806 04:38:33.647310 8669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440: No bootstrap required, opened a new log
I20260806 04:38:33.650849 8669 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb4300b24764681a185da5a4c6e6440" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36649 } }
I20260806 04:38:33.651086 8669 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:33.651147 8669 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb4300b24764681a185da5a4c6e6440, State: Initialized, Role: FOLLOWER
I20260806 04:38:33.651533 8669 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb4300b24764681a185da5a4c6e6440" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36649 } }
I20260806 04:38:33.651687 8669 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:33.651754 8669 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:33.651844 8669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:33.652808 8669 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb4300b24764681a185da5a4c6e6440" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36649 } }
I20260806 04:38:33.653129 8669 leader_election.cc:302] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffb4300b24764681a185da5a4c6e6440; no voters:
I20260806 04:38:33.653399 8669 leader_election.cc:288] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:33.653522 8675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:33.653755 8675 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [term 1 LEADER]: Becoming Leader. State: Replica: ffb4300b24764681a185da5a4c6e6440, State: Running, Role: LEADER
I20260806 04:38:33.654052 8675 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb4300b24764681a185da5a4c6e6440" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36649 } }
I20260806 04:38:33.654549 8669 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:33.655643 8676 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffb4300b24764681a185da5a4c6e6440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb4300b24764681a185da5a4c6e6440" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36649 } } }
I20260806 04:38:33.655812 8676 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:33.655659 8679 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffb4300b24764681a185da5a4c6e6440. Latest consensus state: current_term: 1 leader_uuid: "ffb4300b24764681a185da5a4c6e6440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffb4300b24764681a185da5a4c6e6440" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 36649 } } }
I20260806 04:38:33.655920 8679 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:33.656373 8691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:33.659587 8691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:33.664508 8691 catalog_manager.cc:1347] Generated new cluster ID: 06cf946d2d0e4981a5dfa325e6208f12
I20260806 04:38:33.664578 8691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:33.673681 8691 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:33.674856 8691 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:33.681123 8691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440: Generated new TSK 0
I20260806 04:38:33.681934 8691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:33.685573 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36649
--builtin_ntp_servers=127.12.15.212:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 04:38:33.797662 8708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:33.797915 8708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:33.797952 8708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:33.801909 8708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:33.801968 8708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:33.802028 8708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:33.806710 8708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:33.807801 8708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:33.809288 8708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:38:33.818809 8708 server_base.cc:1034] running on GCE node
W20260806 04:38:33.818759 8718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:33.818756 8720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:33.818756 8716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:33.819363 8708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:33.819977 8708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:33.821163 8708 hybrid_clock.cc:648] HybridClock initialized: now 1785991113821143 us; error 30 us; skew 500 ppm
I20260806 04:38:33.822973 8708 webserver.cc:460] Webserver started at http://127.12.15.193:39303/ using document root <none> and password file <none>
I20260806 04:38:33.823518 8708 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:33.823583 8708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:33.823787 8708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:33.825846 8708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "69bf75664aa84e3b8e2a96e2f64ec878"
format_stamp: "Formatted at 2026-08-06 04:38:33 on dist-test-slave-xt4k"
server_key: "4dc304d60ea92d07f8baf595c36d3e1e"
server_key_iv: "13954c47058a1debf4b8bec843862f7d"
server_key_version: "encryptionkey@0"
I20260806 04:38:33.826393 8708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "69bf75664aa84e3b8e2a96e2f64ec878"
format_stamp: "Formatted at 2026-08-06 04:38:33 on dist-test-slave-xt4k"
server_key: "4dc304d60ea92d07f8baf595c36d3e1e"
server_key_iv: "13954c47058a1debf4b8bec843862f7d"
server_key_version: "encryptionkey@0"
I20260806 04:38:33.830335 8708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:38:33.832890 8729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:33.834182 8708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:33.834300 8708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "69bf75664aa84e3b8e2a96e2f64ec878"
format_stamp: "Formatted at 2026-08-06 04:38:33 on dist-test-slave-xt4k"
server_key: "4dc304d60ea92d07f8baf595c36d3e1e"
server_key_iv: "13954c47058a1debf4b8bec843862f7d"
server_key_version: "encryptionkey@0"
I20260806 04:38:33.834400 8708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:33.847612 8708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:33.848268 8708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:33.848459 8708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:33.849035 8708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:33.850014 8708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:33.850061 8708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:33.850124 8708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:33.850149 8708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:33.859210 8708 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:38269
I20260806 04:38:33.859251 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:38269 every 8 connection(s)
I20260806 04:38:33.860144 8708 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:33.862630 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 8708
I20260806 04:38:33.862782 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:38:33.863087 12351 external_mini_cluster.cc:1428] Setting key 67e92efc2483072dd290dfbfe9471434
I20260806 04:38:33.865339 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36649
--builtin_ntp_servers=127.12.15.212:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 04:38:33.870056 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.862011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:39803 (local address 127.12.15.254:36649)
0806 04:38:33.862193 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:33.862197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:33.863388 (+ 1191us) server_negotiation.cc:407] Connection header received
0806 04:38:33.864937 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:33.864940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:33.864997 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:33.865105 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:33.866030 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.866763 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.867895 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.868219 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.868354 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.869605 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:33.869608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:33.869610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:33.869624 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:33.869645 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:33.869655 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:33.869718 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:33.869877 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 04:38:33.869932 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 04:38:33.871336 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:33.862442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36649 (local address 127.12.15.193:39803)
0806 04:38:33.863218 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:33.863271 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:33.864725 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:33.865258 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:33.865264 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:33.865352 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:33.865883 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:33.865896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.866882 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.866885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:33.867771 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:33.867782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:33.868388 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:33.868393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:33.868584 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:33.869261 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:33.869263 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:33.869317 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:33.869403 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:33.869738 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:33.869748 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:33.869754 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:38:33.869878 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 04:38:33.870215 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260806 04:38:33.872599 8907 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36649
I20260806 04:38:33.872841 8907 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:33.873390 8907 heartbeater.cc:511] Master 127.12.15.254:36649 requested a full tablet report, sending...
I20260806 04:38:33.875066 8617 ts_manager.cc:194] Registered new tserver with Master: 69bf75664aa84e3b8e2a96e2f64ec878 (127.12.15.193:38269)
I20260806 04:38:33.876268 8617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.193:39803
W20260806 04:38:33.980731 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:33.980991 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:33.981029 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:33.984992 8912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:33.985049 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:33.985102 8912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:33.990067 8912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:33.991199 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:33.992586 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:38:34.002256 8912 server_base.cc:1034] running on GCE node
W20260806 04:38:34.002216 8923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:34.002213 8920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:34.002213 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:34.002912 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:34.003567 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:34.004739 8912 hybrid_clock.cc:648] HybridClock initialized: now 1785991114004719 us; error 35 us; skew 500 ppm
I20260806 04:38:34.006597 8912 webserver.cc:460] Webserver started at http://127.12.15.194:32929/ using document root <none> and password file <none>
I20260806 04:38:34.007197 8912 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:34.007267 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:34.007467 8912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:34.009855 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "d57097ae813448fda9cf4d5e5b42710b"
format_stamp: "Formatted at 2026-08-06 04:38:34 on dist-test-slave-xt4k"
server_key: "a9ee8631a12b0455cf978a16cff2ab95"
server_key_iv: "2b901cb1c6967288b5605957c282b91b"
server_key_version: "encryptionkey@0"
I20260806 04:38:34.010550 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "d57097ae813448fda9cf4d5e5b42710b"
format_stamp: "Formatted at 2026-08-06 04:38:34 on dist-test-slave-xt4k"
server_key: "a9ee8631a12b0455cf978a16cff2ab95"
server_key_iv: "2b901cb1c6967288b5605957c282b91b"
server_key_version: "encryptionkey@0"
I20260806 04:38:34.014523 8912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:38:34.017148 8935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:34.018380 8912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:34.018501 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "d57097ae813448fda9cf4d5e5b42710b"
format_stamp: "Formatted at 2026-08-06 04:38:34 on dist-test-slave-xt4k"
server_key: "a9ee8631a12b0455cf978a16cff2ab95"
server_key_iv: "2b901cb1c6967288b5605957c282b91b"
server_key_version: "encryptionkey@0"
I20260806 04:38:34.018610 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:34.046478 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:34.047195 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:34.047376 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:34.047995 8912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:34.048990 8912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:34.049038 8912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:34.049090 8912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:34.049121 8912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:34.057822 8912 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:44721
I20260806 04:38:34.057875 9113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:44721 every 8 connection(s)
I20260806 04:38:34.058717 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:34.064071 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 8912
I20260806 04:38:34.064205 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:38:34.064517 12351 external_mini_cluster.cc:1428] Setting key 83c4ac1b8b012e7fe5bda03ce5d881bf
I20260806 04:38:34.066412 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36649
--builtin_ntp_servers=127.12.15.212:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 04:38:34.068259 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.060466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:37323 (local address 127.12.15.254:36649)
0806 04:38:34.060594 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.060599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.061657 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 04:38:34.063067 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.063070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.063120 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.063193 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:34.064180 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.064820 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.066232 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.066443 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.066545 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.067784 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:34.067788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:34.067790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:34.067802 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:34.067823 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:34.067831 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:34.067899 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.068052 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.068107 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:38:34.069177 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.060839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36649 (local address 127.12.15.194:37323)
0806 04:38:34.061498 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.061546 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.062852 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.063341 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.063347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.063429 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:34.063992 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.064030 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.064951 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.064954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.066119 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.066130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.066608 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.066612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.066789 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.067446 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:34.067448 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:34.067510 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:34.067601 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:34.067914 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:34.067921 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:34.067927 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.068046 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.068375 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
I20260806 04:38:34.070246 9116 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36649
I20260806 04:38:34.070480 9116 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:34.071028 9116 heartbeater.cc:511] Master 127.12.15.254:36649 requested a full tablet report, sending...
I20260806 04:38:34.072100 8617 ts_manager.cc:194] Registered new tserver with Master: d57097ae813448fda9cf4d5e5b42710b (127.12.15.194:44721)
I20260806 04:38:34.072887 8617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.194:37323
W20260806 04:38:34.185611 9119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:34.185909 9119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:34.185953 9119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:34.191422 9119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:34.191514 9119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:34.191576 9119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:34.196904 9119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:36649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:34.198063 9119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:34.199503 9119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:34.209379 9129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:34.209484 9126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:34.209494 9125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:34.209631 9119 server_base.cc:1034] running on GCE node
I20260806 04:38:34.210232 9119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:34.210850 9119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:34.212039 9119 hybrid_clock.cc:648] HybridClock initialized: now 1785991114212002 us; error 49 us; skew 500 ppm
I20260806 04:38:34.214613 9119 webserver.cc:460] Webserver started at http://127.12.15.195:41247/ using document root <none> and password file <none>
I20260806 04:38:34.215232 9119 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:34.215297 9119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:34.215499 9119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:34.217715 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c"
format_stamp: "Formatted at 2026-08-06 04:38:34 on dist-test-slave-xt4k"
server_key: "8d8b0e6adef98d8704e40fd0ec2c7a8f"
server_key_iv: "0b2196af14b753a910b77b27a6c18f53"
server_key_version: "encryptionkey@0"
I20260806 04:38:34.218282 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c"
format_stamp: "Formatted at 2026-08-06 04:38:34 on dist-test-slave-xt4k"
server_key: "8d8b0e6adef98d8704e40fd0ec2c7a8f"
server_key_iv: "0b2196af14b753a910b77b27a6c18f53"
server_key_version: "encryptionkey@0"
I20260806 04:38:34.222921 9119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 04:38:34.226068 9136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:34.227347 9119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 04:38:34.227466 9119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c"
format_stamp: "Formatted at 2026-08-06 04:38:34 on dist-test-slave-xt4k"
server_key: "8d8b0e6adef98d8704e40fd0ec2c7a8f"
server_key_iv: "0b2196af14b753a910b77b27a6c18f53"
server_key_version: "encryptionkey@0"
I20260806 04:38:34.227573 9119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:34.242738 9119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:34.243448 9119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:34.243633 9119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:34.244302 9119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:34.245357 9119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:34.245415 9119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:34.245472 9119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:34.245497 9119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:34.253908 9119 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:37469
I20260806 04:38:34.253938 9319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:37469 every 8 connection(s)
I20260806 04:38:34.254885 9119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:38:34.264576 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 9119
I20260806 04:38:34.264717 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:38:34.265062 12351 external_mini_cluster.cc:1428] Setting key a7a12440f4d3a7ad2ece25fac60650a5
I20260806 04:38:34.265690 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.256823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:53077 (local address 127.12.15.254:36649)
0806 04:38:34.256993 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.256998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.258108 (+ 1110us) server_negotiation.cc:407] Connection header received
0806 04:38:34.259495 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.259499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.259554 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.259633 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:34.261142 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.261829 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.263337 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.263584 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.263687 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.265043 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:34.265046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:34.265048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:34.265063 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:34.265084 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:34.265092 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:34.265181 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.265433 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.265521 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:38:34.267056 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.257272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:36649 (local address 127.12.15.195:53077)
0806 04:38:34.257954 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.257993 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.259279 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.260062 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.260074 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.260279 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:34.260910 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.260928 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.261989 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.261992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.263129 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.263149 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.263796 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.263801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.263971 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.264671 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:34.264673 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:34.264736 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:34.264832 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:34.265205 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:34.265215 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:34.265222 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.265374 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.265671 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260806 04:38:34.268126 9320 heartbeater.cc:347] Connected to a master server at 127.12.15.254:36649
I20260806 04:38:34.268388 9320 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:34.268940 9320 heartbeater.cc:511] Master 127.12.15.254:36649 requested a full tablet report, sending...
I20260806 04:38:34.270154 8617 ts_manager.cc:194] Registered new tserver with Master: e34e832a7c9b4b12b27cbbc3e13ad63c (127.12.15.195:37469)
I20260806 04:38:34.270787 8617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.195:53077
I20260806 04:38:34.280596 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:38:34.287400 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.283216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.12.15.254:36649)
0806 04:38:34.283511 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.283516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.283656 (+ 140us) server_negotiation.cc:407] Connection header received
0806 04:38:34.283727 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.283730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.283781 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.283856 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:34.284174 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.284775 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.285678 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.285891 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.285981 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.286881 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:34.286885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:34.286887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:34.286899 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:34.286918 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:34.286926 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:34.287034 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.287201 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.287243 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 04:38:34.290832 8617 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:38:34.294284 8617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:38:34.311893 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.305764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44721 (local address 127.0.0.1:40814)
0806 04:38:34.306068 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.306081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.306162 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.307040 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.307047 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.307069 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:34.307240 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.307250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.308330 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.308353 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.309795 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.309807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.311245 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.311249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.311355 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.311403 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.311513 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.311735 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":80,"thread_start_us":121,"threads_started":1}
I20260806 04:38:34.311920 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.305953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.12.15.194:44721)
0806 04:38:34.306533 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.306545 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.306572 (+ 27us) server_negotiation.cc:407] Connection header received
0806 04:38:34.306652 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.306658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.306870 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.307051 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:34.307346 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.308227 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.309934 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.311102 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.311219 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.311430 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.311545 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.311654 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":132,"threads_started":1}
I20260806 04:38:34.312525 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.305242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.12.15.193:38269)
0806 04:38:34.306681 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.306692 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.306712 (+ 20us) server_negotiation.cc:407] Connection header received
0806 04:38:34.306776 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.306780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.306918 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.307232 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:34.307487 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.308694 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.310088 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.311228 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.311821 (+ 593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.312184 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.312239 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.312369 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1342,"thread_start_us":172,"threads_started":1}
I20260806 04:38:34.312541 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.305046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38269 (local address 127.0.0.1:53566)
0806 04:38:34.305649 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.305689 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.305825 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.307063 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.307066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.307084 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:34.307380 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.307387 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.308825 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.308828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.309960 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.309971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.311363 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.311367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.311455 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.311479 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.311542 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.312407 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":901,"thread_start_us":91,"threads_started":1}
I20260806 04:38:34.312829 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.306001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.12.15.195:37469)
0806 04:38:34.306399 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.306411 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.306758 (+ 347us) server_negotiation.cc:407] Connection header received
0806 04:38:34.306867 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.306871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.307033 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.307170 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:34.307444 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.308694 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.310767 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.311319 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.311412 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.311603 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.311890 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.312448 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":100,"threads_started":1}
I20260806 04:38:34.313138 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.305768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:37469 (local address 127.0.0.1:45314)
0806 04:38:34.306666 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.306676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.306765 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.307219 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.307221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.307238 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:34.307314 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.307320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.308825 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.308828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.310650 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.310659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.311658 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.311662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.311743 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.311763 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.312982 (+ 1219us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.313048 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":534,"thread_start_us":87,"threads_started":1}
I20260806 04:38:34.315604 8803 tablet_service.cc:1467] Processing CreateTablet for tablet ee940498a95344439994409a74cd4b75 (DEFAULT_TABLE table=test-table [id=42af40654eb04b839b9040b24c87135d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:34.315498 9004 tablet_service.cc:1467] Processing CreateTablet for tablet ee940498a95344439994409a74cd4b75 (DEFAULT_TABLE table=test-table [id=42af40654eb04b839b9040b24c87135d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:34.316608 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee940498a95344439994409a74cd4b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:34.316629 9004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee940498a95344439994409a74cd4b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:34.316533 9212 tablet_service.cc:1467] Processing CreateTablet for tablet ee940498a95344439994409a74cd4b75 (DEFAULT_TABLE table=test-table [id=42af40654eb04b839b9040b24c87135d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:34.317505 9212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee940498a95344439994409a74cd4b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:34.323415 9346 tablet_bootstrap.cc:492] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: Bootstrap starting.
I20260806 04:38:34.324823 9350 tablet_bootstrap.cc:492] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Bootstrap starting.
I20260806 04:38:34.325284 9347 tablet_bootstrap.cc:492] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Bootstrap starting.
I20260806 04:38:34.326037 9346 tablet_bootstrap.cc:654] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:34.326921 9350 tablet_bootstrap.cc:654] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:34.326923 9346 log.cc:824] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:34.326965 9347 tablet_bootstrap.cc:654] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:34.327669 9350 log.cc:824] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:34.327736 9347 log.cc:824] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:34.329262 9346 tablet_bootstrap.cc:492] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: No bootstrap required, opened a new log
I20260806 04:38:34.329479 9346 ts_tablet_manager.cc:1397] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 04:38:34.329558 9350 tablet_bootstrap.cc:492] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: No bootstrap required, opened a new log
I20260806 04:38:34.329732 9350 ts_tablet_manager.cc:1397] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:38:34.329768 9347 tablet_bootstrap.cc:492] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: No bootstrap required, opened a new log
I20260806 04:38:34.329939 9347 ts_tablet_manager.cc:1397] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:38:34.333269 9347 raft_consensus.cc:348] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.333511 9347 raft_consensus.cc:374] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:34.333565 9347 raft_consensus.cc:729] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d57097ae813448fda9cf4d5e5b42710b, State: Initialized, Role: FOLLOWER
I20260806 04:38:34.333835 9350 raft_consensus.cc:348] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.333837 9346 raft_consensus.cc:348] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.333997 9347 consensus_queue.cc:260] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.334084 9350 raft_consensus.cc:374] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:34.334084 9346 raft_consensus.cc:374] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:34.334141 9350 raft_consensus.cc:729] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34e832a7c9b4b12b27cbbc3e13ad63c, State: Initialized, Role: FOLLOWER
I20260806 04:38:34.334141 9346 raft_consensus.cc:729] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69bf75664aa84e3b8e2a96e2f64ec878, State: Initialized, Role: FOLLOWER
I20260806 04:38:34.334553 9346 consensus_queue.cc:260] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.334659 9116 heartbeater.cc:503] Master 127.12.15.254:36649 was elected leader, sending a full tablet report...
I20260806 04:38:34.335024 9347 ts_tablet_manager.cc:1428] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:38:34.335176 8907 heartbeater.cc:503] Master 127.12.15.254:36649 was elected leader, sending a full tablet report...
I20260806 04:38:34.335776 9350 consensus_queue.cc:260] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.336550 9346 ts_tablet_manager.cc:1428] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 04:38:34.336759 9320 heartbeater.cc:503] Master 127.12.15.254:36649 was elected leader, sending a full tablet report...
I20260806 04:38:34.337688 9350 ts_tablet_manager.cc:1428] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260806 04:38:34.361361 8910 tablet.cc:2369] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:34.382289 9358 raft_consensus.cc:482] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:34.382516 9358 raft_consensus.cc:504] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.384003 9358 leader_election.cc:288] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e34e832a7c9b4b12b27cbbc3e13ad63c (127.12.15.195:37469), d57097ae813448fda9cf4d5e5b42710b (127.12.15.194:44721)
I20260806 04:38:34.388936 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.384813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:54683 (local address 127.12.15.195:37469)
0806 04:38:34.384962 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.384966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.385572 (+ 606us) server_negotiation.cc:407] Connection header received
0806 04:38:34.385731 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.385735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.385784 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.385874 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:34.386352 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.386872 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.387934 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.388324 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.388451 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.388502 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.388732 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.388809 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 04:38:34.388959 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.384505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:50723 (local address 127.12.15.194:44721)
0806 04:38:34.384631 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.384636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.385290 (+ 654us) server_negotiation.cc:407] Connection header received
0806 04:38:34.385481 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.385488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.385575 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.385662 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:34.386017 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.386825 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.387964 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.388422 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.388531 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.388581 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.388772 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.388845 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:38:34.388963 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.384413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:44721 (local address 127.12.15.193:50723)
0806 04:38:34.385109 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.385127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.385247 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.385731 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.385739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.385762 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:34.385904 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.385912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.386949 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.386952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.387802 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.387812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.388556 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.388559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.388653 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.388670 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.388744 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.388807 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260806 04:38:34.388968 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.384513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:37469 (local address 127.12.15.193:54683)
0806 04:38:34.385393 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.385476 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:34.385596 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:34.385963 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:34.385968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:34.386002 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:34.386178 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.386185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.386992 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.386996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.387811 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:34.387821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.388463 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.388467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:34.388609 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.388635 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:38:34.388696 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 04:38:34.388771 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":29,"thread_start_us":120,"threads_started":1}
I20260806 04:38:34.389627 9044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee940498a95344439994409a74cd4b75" candidate_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d57097ae813448fda9cf4d5e5b42710b" is_pre_election: true
I20260806 04:38:34.389865 9243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee940498a95344439994409a74cd4b75" candidate_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" is_pre_election: true
I20260806 04:38:34.389981 9044 raft_consensus.cc:2454] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69bf75664aa84e3b8e2a96e2f64ec878 in term 0.
I20260806 04:38:34.390108 9243 raft_consensus.cc:2454] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69bf75664aa84e3b8e2a96e2f64ec878 in term 0.
I20260806 04:38:34.390506 8737 leader_election.cc:302] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69bf75664aa84e3b8e2a96e2f64ec878, e34e832a7c9b4b12b27cbbc3e13ad63c; no voters:
I20260806 04:38:34.390800 9358 raft_consensus.cc:2781] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:38:34.390887 9358 raft_consensus.cc:482] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:38:34.390921 9358 raft_consensus.cc:3037] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:34.392016 9358 raft_consensus.cc:504] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.392395 9358 leader_election.cc:288] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [CANDIDATE]: Term 1 election: Requested vote from peers e34e832a7c9b4b12b27cbbc3e13ad63c (127.12.15.195:37469), d57097ae813448fda9cf4d5e5b42710b (127.12.15.194:44721)
I20260806 04:38:34.392766 9243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee940498a95344439994409a74cd4b75" candidate_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c"
I20260806 04:38:34.392766 9044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee940498a95344439994409a74cd4b75" candidate_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d57097ae813448fda9cf4d5e5b42710b"
I20260806 04:38:34.392906 9243 raft_consensus.cc:3037] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:34.392897 9044 raft_consensus.cc:3037] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:34.394150 9243 raft_consensus.cc:2454] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69bf75664aa84e3b8e2a96e2f64ec878 in term 1.
I20260806 04:38:34.394150 9044 raft_consensus.cc:2454] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69bf75664aa84e3b8e2a96e2f64ec878 in term 1.
I20260806 04:38:34.394451 8737 leader_election.cc:302] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69bf75664aa84e3b8e2a96e2f64ec878, e34e832a7c9b4b12b27cbbc3e13ad63c; no voters:
I20260806 04:38:34.394667 9358 raft_consensus.cc:2781] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:34.394872 9358 raft_consensus.cc:686] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 1 LEADER]: Becoming Leader. State: Replica: 69bf75664aa84e3b8e2a96e2f64ec878, State: Running, Role: LEADER
I20260806 04:38:34.395196 9358 consensus_queue.cc:237] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } }
I20260806 04:38:34.398454 8616 catalog_manager.cc:5495] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69bf75664aa84e3b8e2a96e2f64ec878 (127.12.15.193). New cstate: current_term: 1 leader_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69bf75664aa84e3b8e2a96e2f64ec878" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 } health_report { overall_health: UNKNOWN } } }
I20260806 04:38:34.412161 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.409042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.12.15.193:38269)
0806 04:38:34.409170 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:34.409173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:34.409186 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:38:34.409297 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:34.409300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:34.409351 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:34.409433 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:38:34.409606 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.410080 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.410885 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:34.411088 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:34.411168 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:34.411409 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:38:34.411795 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:38:34.411902 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:34.411980 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 04:38:34.412011 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 04:38:34.417958 9243 raft_consensus.cc:1261] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 1 FOLLOWER]: Refusing update from remote peer 69bf75664aa84e3b8e2a96e2f64ec878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:34.417958 9044 raft_consensus.cc:1261] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 1 FOLLOWER]: Refusing update from remote peer 69bf75664aa84e3b8e2a96e2f64ec878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:34.418637 9363 consensus_queue.cc:1035] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e34e832a7c9b4b12b27cbbc3e13ad63c" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:34.418812 9358 consensus_queue.cc:1035] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d57097ae813448fda9cf4d5e5b42710b" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 44721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:34.435000 8616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:34.435163 8616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:34.437487 8616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffb4300b24764681a185da5a4c6e6440: Sending DeleteTablet for 3 replicas of tablet ee940498a95344439994409a74cd4b75
I20260806 04:38:34.438285 8803 tablet_service.cc:1514] Processing DeleteTablet for tablet ee940498a95344439994409a74cd4b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:34 UTC) from {username='slave'} at 127.0.0.1:53566
I20260806 04:38:34.438411 9004 tablet_service.cc:1514] Processing DeleteTablet for tablet ee940498a95344439994409a74cd4b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:34 UTC) from {username='slave'} at 127.0.0.1:40814
I20260806 04:38:34.438973 9212 tablet_service.cc:1514] Processing DeleteTablet for tablet ee940498a95344439994409a74cd4b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:34 UTC) from {username='slave'} at 127.0.0.1:45314
I20260806 04:38:34.439189 9385 tablet_replica.cc:330] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878: stopping tablet replica
I20260806 04:38:34.439430 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 8708
I20260806 04:38:34.439519 9385 raft_consensus.cc:2229] T ee940498a95344439994409a74cd4b75 P 69bf75664aa84e3b8e2a96e2f64ec878 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:38:34.440086 9386 tablet_replica.cc:330] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: stopping tablet replica
I20260806 04:38:34.440392 9386 raft_consensus.cc:2229] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:34.440663 9386 raft_consensus.cc:2258] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:34.445698 9387 tablet_replica.cc:330] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: stopping tablet replica
I20260806 04:38:34.445997 9387 raft_consensus.cc:2229] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:34.446309 9387 raft_consensus.cc:2258] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:34.448244 9386 ts_tablet_manager.cc:1905] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:34.448534 9387 ts_tablet_manager.cc:1905] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:34.450589 9386 ts_tablet_manager.cc:1918] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:34.450683 9386 log.cc:1196] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/ee940498a95344439994409a74cd4b75
I20260806 04:38:34.450950 9386 ts_tablet_manager.cc:1939] T ee940498a95344439994409a74cd4b75 P d57097ae813448fda9cf4d5e5b42710b: Deleting consensus metadata
I20260806 04:38:34.451120 9387 ts_tablet_manager.cc:1918] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:34.451211 9387 log.cc:1196] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/ee940498a95344439994409a74cd4b75
I20260806 04:38:34.451493 9387 ts_tablet_manager.cc:1939] T ee940498a95344439994409a74cd4b75 P e34e832a7c9b4b12b27cbbc3e13ad63c: Deleting consensus metadata
I20260806 04:38:34.452241 8594 catalog_manager.cc:4841] TS d57097ae813448fda9cf4d5e5b42710b (127.12.15.194:44721): tablet ee940498a95344439994409a74cd4b75 (table test-table [id=42af40654eb04b839b9040b24c87135d]) successfully deleted
I20260806 04:38:34.452641 8597 catalog_manager.cc:4841] TS e34e832a7c9b4b12b27cbbc3e13ad63c (127.12.15.195:37469): tablet ee940498a95344439994409a74cd4b75 (table test-table [id=42af40654eb04b839b9040b24c87135d]) successfully deleted
W20260806 04:38:34.454434 8597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:38269 (error 108)
I20260806 04:38:34.455212 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 8912
I20260806 04:38:34.455554 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:34.455215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:38269 (local address 127.0.0.1:53586)
0806 04:38:34.455368 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:34.455441 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 04:38:34.463521 8597 catalog_manager.cc:4584] TS 69bf75664aa84e3b8e2a96e2f64ec878 (127.12.15.193:38269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee940498a95344439994409a74cd4b75: Network error: Client connection negotiation failed: client connection to 127.12.15.193:38269: connect: Connection refused (error 111)
I20260806 04:38:34.470319 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 9119
I20260806 04:38:34.480361 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 8567
2026-08-06T04:38:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1053 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:34 dist-test-slave-xt4k krb5kdc[9395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:34 dist-test-slave-xt4k krb5kdc[9395](info): set up 2 sockets
Aug 06 04:38:34 dist-test-slave-xt4k krb5kdc[9395](info): commencing operation
krb5kdc: starting...
W20260806 04:38:35.616285 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:35 dist-test-slave-xt4k krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:38:35.739240 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39279
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:39279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:35.852484 9412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:35.852738 9412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:35.852785 9412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:35.856755 9412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:35.856804 9412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:35.856822 9412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:35.856837 9412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:35.856865 9412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:35.862021 9412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:39279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:39279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:35.863059 9412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:35.864051 9412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:35.872536 9420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:35.872542 9418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:35.872602 9422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:35.872735 9412 server_base.cc:1034] running on GCE node
I20260806 04:38:35.873334 9412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:35.874389 9412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:35.875566 9412 hybrid_clock.cc:648] HybridClock initialized: now 1785991115875542 us; error 35 us; skew 500 ppm
Aug 06 04:38:35 dist-test-slave-xt4k krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991115, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:35.878206 9412 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:38:35.879105 9412 webserver.cc:460] Webserver started at http://127.12.15.254:37801/ using document root <none> and password file <none>
I20260806 04:38:35.879644 9412 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:35.879704 9412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:35.879917 9412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:35.881997 9412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "160fb59a175a4f7bac4fc79f5be6900c"
format_stamp: "Formatted at 2026-08-06 04:38:35 on dist-test-slave-xt4k"
server_key: "18105afa274990691dbd318edaf566cb"
server_key_iv: "3f9aa9133db370fd27306fb14497233a"
server_key_version: "encryptionkey@0"
I20260806 04:38:35.882550 9412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "160fb59a175a4f7bac4fc79f5be6900c"
format_stamp: "Formatted at 2026-08-06 04:38:35 on dist-test-slave-xt4k"
server_key: "18105afa274990691dbd318edaf566cb"
server_key_iv: "3f9aa9133db370fd27306fb14497233a"
server_key_version: "encryptionkey@0"
I20260806 04:38:35.886276 9412 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:38:35.888662 9436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:35.889972 9412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:38:35.890098 9412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "160fb59a175a4f7bac4fc79f5be6900c"
format_stamp: "Formatted at 2026-08-06 04:38:35 on dist-test-slave-xt4k"
server_key: "18105afa274990691dbd318edaf566cb"
server_key_iv: "3f9aa9133db370fd27306fb14497233a"
server_key_version: "encryptionkey@0"
I20260806 04:38:35.890211 9412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:35.917069 9412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:35.920109 9412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:35.920284 9412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:35.927297 9412 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:39279
I20260806 04:38:35.927310 9521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:39279 every 8 connection(s)
I20260806 04:38:35.928239 9412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:35.930974 9522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:35.936076 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 9412
I20260806 04:38:35.936204 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:35.936492 12351 external_mini_cluster.cc:1428] Setting key 323a70d00d63ba4337971ba4f0df4ce1
I20260806 04:38:35.936501 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c: Bootstrap starting.
I20260806 04:38:35.939093 9522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:35.939963 9522 log.cc:824] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:35.941859 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c: No bootstrap required, opened a new log
Aug 06 04:38:35 dist-test-slave-xt4k krb5kdc[9395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:35.945503 9522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160fb59a175a4f7bac4fc79f5be6900c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39279 } }
I20260806 04:38:35.945768 9522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:35.945837 9522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160fb59a175a4f7bac4fc79f5be6900c, State: Initialized, Role: FOLLOWER
I20260806 04:38:35.946276 9522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160fb59a175a4f7bac4fc79f5be6900c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39279 } }
I20260806 04:38:35.946408 9522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:35.946475 9522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:35.946574 9522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:35.947407 9522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160fb59a175a4f7bac4fc79f5be6900c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39279 } }
I20260806 04:38:35.947723 9522 leader_election.cc:302] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 160fb59a175a4f7bac4fc79f5be6900c; no voters:
I20260806 04:38:35.948001 9522 leader_election.cc:288] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:35.948101 9530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:35.948305 9530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [term 1 LEADER]: Becoming Leader. State: Replica: 160fb59a175a4f7bac4fc79f5be6900c, State: Running, Role: LEADER
I20260806 04:38:35.949302 9522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:35.950083 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:35.937814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49056 (local address 127.12.15.254:39279)
0806 04:38:35.938232 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:35.938249 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:35.938285 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:38:35.938928 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:35.938945 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:35.939293 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:35.939579 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:35.940497 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:35.941448 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:35.942317 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:35.942787 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:35.942931 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:35.945287 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:35.945292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:35.945305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:35.945331 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:35.947031 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:35.947429 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:35.947436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:35.947442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:35.947569 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:35.947813 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:35.947816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:35.947818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:35.948332 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:35.948562 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:35.949053 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 04:38:35.949280 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
I20260806 04:38:35.951258 9530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160fb59a175a4f7bac4fc79f5be6900c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39279 } }
I20260806 04:38:35.952805 9532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 160fb59a175a4f7bac4fc79f5be6900c. Latest consensus state: current_term: 1 leader_uuid: "160fb59a175a4f7bac4fc79f5be6900c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160fb59a175a4f7bac4fc79f5be6900c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39279 } } }
I20260806 04:38:35.952907 9532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:35.953302 9547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:35.953455 9531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "160fb59a175a4f7bac4fc79f5be6900c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160fb59a175a4f7bac4fc79f5be6900c" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 39279 } } }
I20260806 04:38:35.953572 9531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:35.956698 9547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:35.961807 9547 catalog_manager.cc:1347] Generated new cluster ID: e44a0c46a5744a81ad40462fdd62a616
I20260806 04:38:35.961867 9547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:35.969772 9547 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:35.970969 9547 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:35.976651 9547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c: Generated new TSK 0
I20260806 04:38:35.977337 9547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
Entry for principal kudu/127.12.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab.
I20260806 04:38:36.009101 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39279
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:36.122376 9560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:36.122669 9560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:36.122712 9560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:36.126732 9560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:36.126798 9560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:36.126856 9560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:36.132114 9560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.193.keytab
--principal=kudu/127.12.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:36.133213 9560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:36.134220 9560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:36.143602 9570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:36.143606 9572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:36.143808 9560 server_base.cc:1034] running on GCE node
W20260806 04:38:36.143951 9569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:36.144353 9560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:36.145001 9560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:36.146180 9560 hybrid_clock.cc:648] HybridClock initialized: now 1785991116146162 us; error 27 us; skew 500 ppm
Aug 06 04:38:36 dist-test-slave-xt4k krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:36.148922 9560 init.cc:377] Logged in from keytab as kudu/127.12.15.193@KRBTEST.COM (short username kudu)
I20260806 04:38:36.149806 9560 webserver.cc:460] Webserver started at http://127.12.15.193:42721/ using document root <none> and password file <none>
I20260806 04:38:36.150375 9560 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:36.150435 9560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:36.150636 9560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:36.152693 9560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "34e8adc668b0471aa9465325d61d77bc"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "caa561bbe4b2070f5ab21ebbf52e8b7b"
server_key_iv: "a9f2e82b58a81572e499d714fbf30015"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.153205 9560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "34e8adc668b0471aa9465325d61d77bc"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "caa561bbe4b2070f5ab21ebbf52e8b7b"
server_key_iv: "a9f2e82b58a81572e499d714fbf30015"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.156771 9560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:38:36.159149 9585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.160447 9560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:36.160564 9560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "34e8adc668b0471aa9465325d61d77bc"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "caa561bbe4b2070f5ab21ebbf52e8b7b"
server_key_iv: "a9f2e82b58a81572e499d714fbf30015"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.160666 9560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:36.189143 9560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:36.192435 9560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:36.192622 9560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:36.193244 9560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:36.194139 9560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:36.194193 9560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.194252 9560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:36.194280 9560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.202940 9560 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:44583
I20260806 04:38:36.202965 9752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:44583 every 8 connection(s)
I20260806 04:38:36.203831 9560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:36.206591 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 9560
I20260806 04:38:36.206784 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:38:36.207175 12351 external_mini_cluster.cc:1428] Setting key e08f4b91ce982d2570983491df04a151
Aug 06 04:38:36 dist-test-slave-xt4k krb5kdc[9395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.193@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.194@KRBTEST.COM; defaulting to no policy
I20260806 04:38:36.218642 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.205723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:55315 (local address 127.12.15.254:39279)
0806 04:38:36.205913 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.205917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.206838 (+ 921us) server_negotiation.cc:407] Connection header received
0806 04:38:36.208365 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.208369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.208417 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.208495 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:36.209692 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.210393 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.212020 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.212249 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.212330 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.215175 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:36.215178 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:36.215180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:36.215207 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:36.216628 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.217195 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.217198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.217199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.217272 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.217617 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.217619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.217620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.217819 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:36.217894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.218395 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.218507 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 04:38:36.219532 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.206076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39279 (local address 127.12.15.193:55315)
0806 04:38:36.206615 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.206675 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.208164 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.208707 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.208715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.209063 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:36.209534 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.209547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.210555 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.210559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.211893 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.211905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.212422 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.212426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.212622 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.213223 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:36.213244 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:36.215001 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:36.216736 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:36.216743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:36.216760 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:36.217106 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:36.217347 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:36.217350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:36.217352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:36.217532 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:36.217920 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:36.217927 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:36.218195 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.218358 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.218677 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.194@KRBTEST.COM" created.
I20260806 04:38:36.220907 9755 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39279
I20260806 04:38:36.221158 9755 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:36.221657 9755 heartbeater.cc:511] Master 127.12.15.254:39279 requested a full tablet report, sending...
I20260806 04:38:36.223363 9474 ts_manager.cc:194] Registered new tserver with Master: 34e8adc668b0471aa9465325d61d77bc (127.12.15.193:44583)
I20260806 04:38:36.224635 9474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.193@KRBTEST.COM'} at 127.12.15.193:55315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
Entry for principal kudu/127.12.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab.
I20260806 04:38:36.235538 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39279
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:36.348353 9761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:36.348625 9761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:36.348667 9761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:36.352813 9761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:36.352864 9761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:36.352921 9761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:36.358038 9761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.194.keytab
--principal=kudu/127.12.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:36.358996 9761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:36.359921 9761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:36.369546 9769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:36.369561 9772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:36.369694 9761 server_base.cc:1034] running on GCE node
W20260806 04:38:36.369583 9775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:36.370168 9761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:36.370800 9761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:36.371971 9761 hybrid_clock.cc:648] HybridClock initialized: now 1785991116371949 us; error 33 us; skew 500 ppm
Aug 06 04:38:36 dist-test-slave-xt4k krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:36.374805 9761 init.cc:377] Logged in from keytab as kudu/127.12.15.194@KRBTEST.COM (short username kudu)
I20260806 04:38:36.375670 9761 webserver.cc:460] Webserver started at http://127.12.15.194:33603/ using document root <none> and password file <none>
I20260806 04:38:36.376204 9761 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:36.376268 9761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:36.376487 9761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:36.378451 9761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "56c39e3ce53844b8909a341e3e89b864"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "2acf82873b3efdd6b584a17987a0f3b7"
server_key_iv: "ad050a5531610c232cf821ccd6a939aa"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.378973 9761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "56c39e3ce53844b8909a341e3e89b864"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "2acf82873b3efdd6b584a17987a0f3b7"
server_key_iv: "ad050a5531610c232cf821ccd6a939aa"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.382500 9761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:38:36.384840 9788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.386013 9761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:36.386147 9761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "56c39e3ce53844b8909a341e3e89b864"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "2acf82873b3efdd6b584a17987a0f3b7"
server_key_iv: "ad050a5531610c232cf821ccd6a939aa"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.386238 9761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:36.395767 9761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:36.398681 9761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:36.398849 9761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:36.399362 9761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:36.400152 9761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:36.400198 9761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.400238 9761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:36.400267 9761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.408232 9761 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:45373
I20260806 04:38:36.408295 9948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:45373 every 8 connection(s)
I20260806 04:38:36.409088 9761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:36.412204 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 9761
I20260806 04:38:36.412380 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:38:36.412724 12351 external_mini_cluster.cc:1428] Setting key 00e5a8ad1114d7fc9fae8b53ad8ad99d
Aug 06 04:38:36 dist-test-slave-xt4k krb5kdc[9395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.194@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:36.423123 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.410743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:38605 (local address 127.12.15.254:39279)
0806 04:38:36.410858 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.410861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.411696 (+ 835us) server_negotiation.cc:407] Connection header received
0806 04:38:36.413354 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.413357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.413406 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.413488 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:36.414850 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.415389 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.416807 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.417155 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.417273 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.419541 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:36.419545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:36.419547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:36.419578 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:36.421084 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.421543 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.421547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.421548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.421649 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.421937 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.421940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.421942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.422175 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:36.422272 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.422819 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.422942 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.12.15.195@KRBTEST.COM; defaulting to no policy
I20260806 04:38:36.423888 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.411087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39279 (local address 127.12.15.194:38605)
0806 04:38:36.411540 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.411585 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.413136 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.413648 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.413655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.414059 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:36.414676 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.414699 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.415517 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.415521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.416690 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.416700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.417295 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.417297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.417435 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.417966 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:36.417982 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:36.419373 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:36.421197 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:36.421202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:36.421214 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:36.421453 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:36.421729 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:36.421732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:36.421733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:36.421856 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:36.422283 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:36.422289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:36.422583 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.422738 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.422977 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260806 04:38:36.425143 9949 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39279
I20260806 04:38:36.425371 9949 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.195@KRBTEST.COM" created.
I20260806 04:38:36.425889 9949 heartbeater.cc:511] Master 127.12.15.254:39279 requested a full tablet report, sending...
I20260806 04:38:36.426994 9474 ts_manager.cc:194] Registered new tserver with Master: 56c39e3ce53844b8909a341e3e89b864 (127.12.15.194:45373)
I20260806 04:38:36.427687 9474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.194@KRBTEST.COM'} at 127.12.15.194:38605
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
Entry for principal kudu/127.12.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab.
I20260806 04:38:36.441630 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39279
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:36.553311 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:36.553560 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:36.553618 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:36.557528 9957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:36.557580 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:36.557657 9957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:36.562738 9957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.195.keytab
--principal=kudu/127.12.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:39279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:36.563717 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:36.564750 9957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:36.574136 9971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:36.574136 9966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:36.574126 9967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:36.574424 9957 server_base.cc:1034] running on GCE node
I20260806 04:38:36.574906 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:36.575541 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:36.576726 9957 hybrid_clock.cc:648] HybridClock initialized: now 1785991116576709 us; error 37 us; skew 500 ppm
Aug 06 04:38:36 dist-test-slave-xt4k krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:36.579490 9957 init.cc:377] Logged in from keytab as kudu/127.12.15.195@KRBTEST.COM (short username kudu)
I20260806 04:38:36.580399 9957 webserver.cc:460] Webserver started at http://127.12.15.195:46013/ using document root <none> and password file <none>
I20260806 04:38:36.580922 9957 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:36.580981 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:36.581154 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:36.583148 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "81c1245579684fd1a3cb31b59f830a01"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "c330c93ba517de3b9a8e40aedac1f095"
server_key_iv: "e09047cf0e626d1386ee2ccb5b7b504f"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.583654 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "81c1245579684fd1a3cb31b59f830a01"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "c330c93ba517de3b9a8e40aedac1f095"
server_key_iv: "e09047cf0e626d1386ee2ccb5b7b504f"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.587296 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:38:36.589747 9979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.591013 9957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:36.591138 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "81c1245579684fd1a3cb31b59f830a01"
format_stamp: "Formatted at 2026-08-06 04:38:36 on dist-test-slave-xt4k"
server_key: "c330c93ba517de3b9a8e40aedac1f095"
server_key_iv: "e09047cf0e626d1386ee2ccb5b7b504f"
server_key_version: "encryptionkey@0"
I20260806 04:38:36.591238 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:36.614637 9957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:36.617626 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:36.617795 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:36.618372 9957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:36.619156 9957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:36.619201 9957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.619240 9957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:36.619259 9957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:36.627626 9957 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:40995
I20260806 04:38:36.627655 10132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:40995 every 8 connection(s)
I20260806 04:38:36.628525 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
Aug 06 04:38:36 dist-test-slave-xt4k krb5kdc[9395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.195@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:36.638269 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 9957
I20260806 04:38:36.638357 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:38:36.638608 12351 external_mini_cluster.cc:1428] Setting key e91ae3118f3df411b0a46a84f0ebdabf
I20260806 04:38:36.641361 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.630199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:34307 (local address 127.12.15.254:39279)
0806 04:38:36.630336 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.630340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.631128 (+ 788us) server_negotiation.cc:407] Connection header received
0806 04:38:36.632440 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.632443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.632499 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.632574 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:36.633675 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.634196 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.635366 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.635630 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.635731 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.638021 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:36.638024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:36.638026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:36.638051 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:36.639368 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.639861 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.639864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.639865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.639945 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.640286 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.640288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.640289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.640526 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:36.640614 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.641084 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.641202 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:38:36.642060 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.630544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:39279 (local address 127.12.15.195:34307)
0806 04:38:36.630987 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.631026 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.632230 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.632746 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.632753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.633069 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:36.633527 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.633540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.634309 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.634312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.635186 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.635195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.635788 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.635793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.635961 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.636471 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:38:36.636487 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:36.637849 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:36.639499 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:36.639505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:36.639518 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:36.639757 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:36.640025 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:38:36.640028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:38:36.640030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:38:36.640202 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:38:36.640631 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:36.640637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:36.640856 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.641014 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.641313 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260806 04:38:36.643208 10133 heartbeater.cc:347] Connected to a master server at 127.12.15.254:39279
I20260806 04:38:36.643414 10133 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:36.643846 10133 heartbeater.cc:511] Master 127.12.15.254:39279 requested a full tablet report, sending...
I20260806 04:38:36.644840 9474 ts_manager.cc:194] Registered new tserver with Master: 81c1245579684fd1a3cb31b59f830a01 (127.12.15.195:40995)
I20260806 04:38:36.645355 9474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.195@KRBTEST.COM'} at 127.12.15.195:34307
I20260806 04:38:36.652310 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:38:36.661209 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.654209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49064 (local address 127.12.15.254:39279)
0806 04:38:36.654339 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.654344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.654516 (+ 172us) server_negotiation.cc:407] Connection header received
0806 04:38:36.654640 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.654644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.654688 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.654747 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:36.655399 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.655907 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.656838 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.657063 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.657150 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.658386 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:36.658390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:36.658393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:36.658421 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:36.659592 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.660005 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.660009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.660010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.660091 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:36.660359 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:36.660363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:36.660364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:36.660584 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:36.660704 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.660949 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.661057 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 04:38:36.663909 9474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:38:36.666908 9474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:38:36.680794 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.675182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:44583 (local address 127.0.0.1:37174)
0806 04:38:36.675547 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.675612 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.675747 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.676521 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.676527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.676552 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:36.676681 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.676689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.677983 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.677986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.679082 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.679092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.680210 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.680213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.680320 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.680390 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.680504 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.680635 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260806 04:38:36.680799 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.675319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.12.15.193:44583)
0806 04:38:36.675678 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.675696 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.675722 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:38:36.675919 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.675925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.676153 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.676310 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:36.676811 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.677871 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.679216 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.680101 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.680211 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.680399 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.680546 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.680651 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":155,"threads_started":1}
I20260806 04:38:36.681049 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.675898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:45373 (local address 127.0.0.1:35416)
0806 04:38:36.676667 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.676680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.676775 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.677200 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.677203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.677217 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:36.677309 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.677314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.678695 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.678698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.679761 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.679771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.680728 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.680732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.680823 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.680847 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.680896 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.680958 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":71,"thread_start_us":98,"threads_started":1}
I20260806 04:38:36.681648 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.676220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.12.15.194:45373)
0806 04:38:36.676592 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.676625 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.676765 (+ 140us) server_negotiation.cc:407] Connection header received
0806 04:38:36.676880 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.676887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.677075 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.677223 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:36.677396 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.678539 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.679869 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.680604 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.681298 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.681408 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.681447 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.681516 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":179,"threads_started":1}
I20260806 04:38:36.682279 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.676303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:40995 (local address 127.0.0.1:46146)
0806 04:38:36.676974 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.676986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.677065 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.677596 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.677599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.677615 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:36.677697 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.677701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.678953 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.678957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.680132 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.680143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.681639 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.681642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.681813 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.681838 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.682055 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.682142 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":21,"thread_start_us":99,"threads_started":1}
I20260806 04:38:36.683910 9662 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc681e59a834131958708d303ff8c67 (DEFAULT_TABLE table=test-table [id=db3fd7fde5574323b8ccabd7ca92d313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:36.684926 9662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fc681e59a834131958708d303ff8c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:36.685503 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.676749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.12.15.195:40995)
0806 04:38:36.677193 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.677204 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.677225 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:38:36.677294 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.677299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.677470 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.677616 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:36.677782 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.678823 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.680264 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.681513 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.681620 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.685182 (+ 3562us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.685255 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.685364 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":136,"threads_started":1}
I20260806 04:38:36.686508 9859 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc681e59a834131958708d303ff8c67 (DEFAULT_TABLE table=test-table [id=db3fd7fde5574323b8ccabd7ca92d313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:36.687629 9859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fc681e59a834131958708d303ff8c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:36.688371 10046 tablet_service.cc:1467] Processing CreateTablet for tablet 7fc681e59a834131958708d303ff8c67 (DEFAULT_TABLE table=test-table [id=db3fd7fde5574323b8ccabd7ca92d313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:36.689461 10046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fc681e59a834131958708d303ff8c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:36.692271 10155 tablet_bootstrap.cc:492] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: Bootstrap starting.
I20260806 04:38:36.694358 10155 tablet_bootstrap.cc:654] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:36.695598 10155 log.cc:824] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:36.695786 10156 tablet_bootstrap.cc:492] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Bootstrap starting.
I20260806 04:38:36.697638 10155 tablet_bootstrap.cc:492] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: No bootstrap required, opened a new log
I20260806 04:38:36.697618 10159 tablet_bootstrap.cc:492] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Bootstrap starting.
I20260806 04:38:36.697829 10155 ts_tablet_manager.cc:1397] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:38:36.698073 10156 tablet_bootstrap.cc:654] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:36.699369 10156 log.cc:824] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:36.699721 10159 tablet_bootstrap.cc:654] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:36.700881 10159 log.cc:824] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:36.700896 10155 raft_consensus.cc:348] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.701148 10155 raft_consensus.cc:374] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:36.701202 10155 raft_consensus.cc:729] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e8adc668b0471aa9465325d61d77bc, State: Initialized, Role: FOLLOWER
I20260806 04:38:36.701478 10156 tablet_bootstrap.cc:492] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: No bootstrap required, opened a new log
I20260806 04:38:36.701687 10156 ts_tablet_manager.cc:1397] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 04:38:36.701632 10155 consensus_queue.cc:260] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.702471 10155 ts_tablet_manager.cc:1428] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 04:38:36.702759 10159 tablet_bootstrap.cc:492] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: No bootstrap required, opened a new log
I20260806 04:38:36.702824 9755 heartbeater.cc:503] Master 127.12.15.254:39279 was elected leader, sending a full tablet report...
I20260806 04:38:36.702931 10159 ts_tablet_manager.cc:1397] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
W20260806 04:38:36.704914 9756 tablet.cc:2369] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:36.705466 10156 raft_consensus.cc:348] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.705725 10156 raft_consensus.cc:374] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:36.705775 10156 raft_consensus.cc:729] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c39e3ce53844b8909a341e3e89b864, State: Initialized, Role: FOLLOWER
I20260806 04:38:36.706214 10156 consensus_queue.cc:260] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.706897 9949 heartbeater.cc:503] Master 127.12.15.254:39279 was elected leader, sending a full tablet report...
I20260806 04:38:36.707249 10156 ts_tablet_manager.cc:1428] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:38:36.706616 10159 raft_consensus.cc:348] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.707471 10159 raft_consensus.cc:374] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:36.707538 10159 raft_consensus.cc:729] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c1245579684fd1a3cb31b59f830a01, State: Initialized, Role: FOLLOWER
I20260806 04:38:36.708065 10159 consensus_queue.cc:260] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.708781 10133 heartbeater.cc:503] Master 127.12.15.254:39279 was elected leader, sending a full tablet report...
I20260806 04:38:36.709035 10159 ts_tablet_manager.cc:1428] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260806 04:38:36.857867 10165 raft_consensus.cc:482] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:36.858094 10165 raft_consensus.cc:504] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.859221 10165 leader_election.cc:288] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e8adc668b0471aa9465325d61d77bc (127.12.15.193:44583), 81c1245579684fd1a3cb31b59f830a01 (127.12.15.195:40995)
I20260806 04:38:36.863520 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.859586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:42907 (local address 127.12.15.193:44583)
0806 04:38:36.859705 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.859710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.859727 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:38:36.859788 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.859791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.859840 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.859923 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:36.860217 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.860970 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.862413 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.862972 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.863097 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.863162 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.863328 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.863406 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 04:38:36.863518 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.859486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:44583 (local address 127.12.15.194:42907)
0806 04:38:36.859599 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.859613 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.859701 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.859955 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.859958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.859986 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:36.860099 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.860105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.861094 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.861097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.862264 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.862276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.863096 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.863099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.863196 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.863226 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.863301 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.863377 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 04:38:36.863740 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.859486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:40995 (local address 127.12.15.194:54701)
0806 04:38:36.860088 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.860099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:36.860194 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:36.860459 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:36.860462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:36.860478 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:36.860607 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.860615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.861430 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.861434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.862530 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:36.862538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.863331 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.863334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:36.863457 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.863479 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 04:38:36.863552 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 04:38:36.863616 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":270,"thread_start_us":129,"threads_started":1}
I20260806 04:38:36.863795 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.859727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:54701 (local address 127.12.15.195:40995)
0806 04:38:36.859878 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.859883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.860183 (+ 300us) server_negotiation.cc:407] Connection header received
0806 04:38:36.860299 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.860303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.860374 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.860441 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:36.860724 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.861317 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.862646 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.863229 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.863326 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.863370 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.863580 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.863656 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 04:38:36.864187 9691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc681e59a834131958708d303ff8c67" candidate_uuid: "56c39e3ce53844b8909a341e3e89b864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e8adc668b0471aa9465325d61d77bc" is_pre_election: true
I20260806 04:38:36.864535 9691 raft_consensus.cc:2454] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c39e3ce53844b8909a341e3e89b864 in term 0.
I20260806 04:38:36.864769 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc681e59a834131958708d303ff8c67" candidate_uuid: "56c39e3ce53844b8909a341e3e89b864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c1245579684fd1a3cb31b59f830a01" is_pre_election: true
I20260806 04:38:36.865068 10070 raft_consensus.cc:2454] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56c39e3ce53844b8909a341e3e89b864 in term 0.
I20260806 04:38:36.865077 9795 leader_election.cc:302] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e8adc668b0471aa9465325d61d77bc, 56c39e3ce53844b8909a341e3e89b864; no voters:
I20260806 04:38:36.865360 10165 raft_consensus.cc:2781] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:38:36.865461 10165 raft_consensus.cc:482] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:38:36.865514 10165 raft_consensus.cc:3037] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:36.866601 10165 raft_consensus.cc:504] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.866910 10165 leader_election.cc:288] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [CANDIDATE]: Term 1 election: Requested vote from peers 34e8adc668b0471aa9465325d61d77bc (127.12.15.193:44583), 81c1245579684fd1a3cb31b59f830a01 (127.12.15.195:40995)
I20260806 04:38:36.867265 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc681e59a834131958708d303ff8c67" candidate_uuid: "56c39e3ce53844b8909a341e3e89b864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c1245579684fd1a3cb31b59f830a01"
I20260806 04:38:36.867277 9691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fc681e59a834131958708d303ff8c67" candidate_uuid: "56c39e3ce53844b8909a341e3e89b864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e8adc668b0471aa9465325d61d77bc"
I20260806 04:38:36.867386 10070 raft_consensus.cc:3037] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:36.867386 9691 raft_consensus.cc:3037] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:36.868649 10070 raft_consensus.cc:2454] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56c39e3ce53844b8909a341e3e89b864 in term 1.
I20260806 04:38:36.868649 9691 raft_consensus.cc:2454] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56c39e3ce53844b8909a341e3e89b864 in term 1.
I20260806 04:38:36.868970 9795 leader_election.cc:302] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e8adc668b0471aa9465325d61d77bc, 56c39e3ce53844b8909a341e3e89b864; no voters:
I20260806 04:38:36.869216 10165 raft_consensus.cc:2781] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:36.869437 10165 raft_consensus.cc:686] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 1 LEADER]: Becoming Leader. State: Replica: 56c39e3ce53844b8909a341e3e89b864, State: Running, Role: LEADER
I20260806 04:38:36.869908 10165 consensus_queue.cc:237] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } }
I20260806 04:38:36.873276 9473 catalog_manager.cc:5495] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56c39e3ce53844b8909a341e3e89b864 (127.12.15.194). New cstate: current_term: 1 leader_uuid: "56c39e3ce53844b8909a341e3e89b864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56c39e3ce53844b8909a341e3e89b864" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 45373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 } health_report { overall_health: UNKNOWN } } }
W20260806 04:38:36.879423 10134 tablet.cc:2369] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:38:36.910187 9952 tablet.cc:2369] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:36.915336 10151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.912069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.12.15.194:45373)
0806 04:38:36.912179 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:36.912183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:36.912195 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:38:36.912311 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:36.912314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:36.912409 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:36.912515 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:38:36.912750 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.913237 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.914113 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:36.914314 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:36.914379 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:36.914627 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:38:36.915029 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:38:36.915117 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:36.915160 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 04:38:36.915212 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 04:38:36.921216 9691 raft_consensus.cc:1261] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 1 FOLLOWER]: Refusing update from remote peer 56c39e3ce53844b8909a341e3e89b864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:36.921214 10070 raft_consensus.cc:1261] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 1 FOLLOWER]: Refusing update from remote peer 56c39e3ce53844b8909a341e3e89b864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:36.921872 10165 consensus_queue.cc:1035] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c1245579684fd1a3cb31b59f830a01" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 40995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:36.922056 10169 consensus_queue.cc:1035] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e8adc668b0471aa9465325d61d77bc" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 44583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:36.929068 10173 mvcc.cc:205] Tried to move back new op lower bound from 7315419614901731328 to 7315419614701895680. Current Snapshot: MvccSnapshot[applied={T|T < 7315419614901731328}]
I20260806 04:38:36.932680 10176 mvcc.cc:205] Tried to move back new op lower bound from 7315419614901731328 to 7315419614701895680. Current Snapshot: MvccSnapshot[applied={T|T < 7315419614901731328 or (T in {7315419614901731328})}]
I20260806 04:38:36.938586 9474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:36.938766 9474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:36.941290 9474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 160fb59a175a4f7bac4fc79f5be6900c: Sending DeleteTablet for 3 replicas of tablet 7fc681e59a834131958708d303ff8c67
I20260806 04:38:36.942080 9662 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc681e59a834131958708d303ff8c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:36 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:37174
I20260806 04:38:36.942194 10046 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc681e59a834131958708d303ff8c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:36 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:46146
I20260806 04:38:36.942565 10187 tablet_replica.cc:330] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc: stopping tablet replica
I20260806 04:38:36.942632 9859 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fc681e59a834131958708d303ff8c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:36 UTC) from {username='kudu', principal='kudu/127.12.15.254@KRBTEST.COM'} at 127.0.0.1:35416
I20260806 04:38:36.942885 10187 raft_consensus.cc:2229] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:36.943104 10190 tablet_replica.cc:330] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: stopping tablet replica
I20260806 04:38:36.943207 10187 raft_consensus.cc:2258] T 7fc681e59a834131958708d303ff8c67 P 34e8adc668b0471aa9465325d61d77bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:36.943480 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 9560
I20260806 04:38:36.943471 10190 raft_consensus.cc:2229] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:38:36.946301 10190 raft_consensus.cc:2258] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:36.948796 10190 ts_tablet_manager.cc:1905] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:36.952724 10190 ts_tablet_manager.cc:1918] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:36.952886 10190 log.cc:1196] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/7fc681e59a834131958708d303ff8c67
I20260806 04:38:36.953270 10190 ts_tablet_manager.cc:1939] T 7fc681e59a834131958708d303ff8c67 P 56c39e3ce53844b8909a341e3e89b864: Deleting consensus metadata
I20260806 04:38:36.954519 9443 catalog_manager.cc:4841] TS 56c39e3ce53844b8909a341e3e89b864 (127.12.15.194:45373): tablet 7fc681e59a834131958708d303ff8c67 (table test-table [id=db3fd7fde5574323b8ccabd7ca92d313]) successfully deleted
I20260806 04:38:36.958622 10188 tablet_replica.cc:330] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: stopping tablet replica
I20260806 04:38:36.958973 10188 raft_consensus.cc:2229] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:36.959316 10188 raft_consensus.cc:2258] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 04:38:36.964514 9443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:44583 (error 108)
I20260806 04:38:36.965322 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 9761
I20260806 04:38:36.965732 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:36.965404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:44583 (local address 127.0.0.1:37178)
0806 04:38:36.965584 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:36.965648 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 04:38:36.967336 10188 ts_tablet_manager.cc:1905] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 04:38:36.969286 9443 catalog_manager.cc:4584] TS 34e8adc668b0471aa9465325d61d77bc (127.12.15.193:44583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7fc681e59a834131958708d303ff8c67: Network error: Client connection negotiation failed: client connection to 127.12.15.193:44583: connect: Connection refused (error 111)
I20260806 04:38:36.970414 10188 ts_tablet_manager.cc:1918] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:36.970515 10188 log.cc:1196] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/7fc681e59a834131958708d303ff8c67
I20260806 04:38:36.970816 10188 ts_tablet_manager.cc:1939] T 7fc681e59a834131958708d303ff8c67 P 81c1245579684fd1a3cb31b59f830a01: Deleting consensus metadata
I20260806 04:38:36.972114 9439 catalog_manager.cc:4841] TS 81c1245579684fd1a3cb31b59f830a01 (127.12.15.195:40995): tablet 7fc681e59a834131958708d303ff8c67 (table test-table [id=db3fd7fde5574323b8ccabd7ca92d313]) successfully deleted
I20260806 04:38:36.975086 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 9957
I20260806 04:38:36.985219 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 9412
2026-08-06T04:38:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2506 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T04:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:37Z Disabled control of system clock
I20260806 04:38:37.018666 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:32997
--webserver_interface=127.12.15.254
--webserver_port=0
--builtin_ntp_servers=127.12.15.212:40953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:32997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 04:38:37.128274 10197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:37.128549 10197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:37.128582 10197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:37.132467 10197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:37.132515 10197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:37.132529 10197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:37.132543 10197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:37.132557 10197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:37.137223 10197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:32997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:32997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:37.138212 10197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:37.139511 10197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:37.147534 10204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:37.147534 10203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:37.147564 10208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:37.147897 10197 server_base.cc:1034] running on GCE node
I20260806 04:38:37.148439 10197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:37.149469 10197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:37.150616 10197 hybrid_clock.cc:648] HybridClock initialized: now 1785991117150590 us; error 38 us; skew 500 ppm
I20260806 04:38:37.155189 10197 webserver.cc:460] Webserver started at http://127.12.15.254:41757/ using document root <none> and password file <none>
I20260806 04:38:37.155756 10197 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:37.155822 10197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:37.156018 10197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:37.158113 10197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "7960c91a9fb648729591441e1767b218"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "f12d7878f19d87930ce60e777d4d75a7"
server_key_iv: "57696fb7f9955135dd26e0ecfcb05fdb"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.158677 10197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "7960c91a9fb648729591441e1767b218"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "f12d7878f19d87930ce60e777d4d75a7"
server_key_iv: "57696fb7f9955135dd26e0ecfcb05fdb"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.162391 10197 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:38:37.164865 10214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.166131 10197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:37.166249 10197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "7960c91a9fb648729591441e1767b218"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "f12d7878f19d87930ce60e777d4d75a7"
server_key_iv: "57696fb7f9955135dd26e0ecfcb05fdb"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.166342 10197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:37.187052 10197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:37.187726 10197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:37.187894 10197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:37.194802 10197 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:32997
I20260806 04:38:37.194895 10286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:32997 every 8 connection(s)
I20260806 04:38:37.195775 10197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:37.198890 10290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:37.204452 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 10197
I20260806 04:38:37.204572 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:37.204653 10290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218: Bootstrap starting.
I20260806 04:38:37.204861 12351 external_mini_cluster.cc:1428] Setting key db075252dbb7adb926cc245d57675f8d
I20260806 04:38:37.207100 10290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:37.207816 10290 log.cc:824] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:37.209746 10290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218: No bootstrap required, opened a new log
I20260806 04:38:37.213826 10290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7960c91a9fb648729591441e1767b218" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32997 } }
I20260806 04:38:37.214017 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.206144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45326 (local address 127.12.15.254:32997)
0806 04:38:37.206754 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.206768 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.206802 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:38:37.207517 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.207538 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.207932 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.208268 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:37.208721 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.209687 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.210875 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.211358 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.211472 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.212295 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:37.212300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:37.212313 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:37.212724 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:37.212779 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:37.212800 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:37.212875 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.213025 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.213301 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260806 04:38:37.214040 10290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:37.214114 10290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7960c91a9fb648729591441e1767b218, State: Initialized, Role: FOLLOWER
I20260806 04:38:37.214566 10290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7960c91a9fb648729591441e1767b218" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32997 } }
I20260806 04:38:37.214692 10290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:37.214752 10290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:37.214851 10290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:37.215966 10290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7960c91a9fb648729591441e1767b218" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32997 } }
I20260806 04:38:37.216317 10290 leader_election.cc:302] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7960c91a9fb648729591441e1767b218; no voters:
I20260806 04:38:37.216692 10290 leader_election.cc:288] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:37.216787 10299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:37.216995 10299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [term 1 LEADER]: Becoming Leader. State: Replica: 7960c91a9fb648729591441e1767b218, State: Running, Role: LEADER
I20260806 04:38:37.217301 10299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7960c91a9fb648729591441e1767b218" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32997 } }
I20260806 04:38:37.217882 10290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:37.218943 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7960c91a9fb648729591441e1767b218. Latest consensus state: current_term: 1 leader_uuid: "7960c91a9fb648729591441e1767b218" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7960c91a9fb648729591441e1767b218" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32997 } } }
I20260806 04:38:37.219082 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:37.219342 10301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7960c91a9fb648729591441e1767b218" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7960c91a9fb648729591441e1767b218" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32997 } } }
I20260806 04:38:37.219480 10301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:37.219596 10312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:37.222630 10312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:37.227634 10312 catalog_manager.cc:1347] Generated new cluster ID: b59a9e817a0545d3a8ed95a904ab37cd
I20260806 04:38:37.227697 10312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:37.244884 10312 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:37.246237 10312 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:37.254998 10312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218: Generated new TSK 0
I20260806 04:38:37.255815 10312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:37.259294 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.193:0
--local_ip_for_outbound_sockets=127.12.15.193
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:32997
--builtin_ntp_servers=127.12.15.212:40953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 04:38:37.370911 10330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:37.371145 10330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:37.371196 10330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:37.375187 10330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:37.375238 10330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:37.375311 10330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.193
I20260806 04:38:37.379889 10330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.193
--webserver_port=0
--tserver_master_addrs=127.12.15.254:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.193
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:37.380857 10330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:37.382158 10330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:38:37.391541 10330 server_base.cc:1034] running on GCE node
W20260806 04:38:37.391564 10342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:37.391535 10344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:37.391705 10339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:37.392145 10330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:37.392786 10330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:37.393975 10330 hybrid_clock.cc:648] HybridClock initialized: now 1785991117393952 us; error 31 us; skew 500 ppm
I20260806 04:38:37.395797 10330 webserver.cc:460] Webserver started at http://127.12.15.193:34977/ using document root <none> and password file <none>
I20260806 04:38:37.396394 10330 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:37.396466 10330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:37.396679 10330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:37.398702 10330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data/instance:
uuid: "3b7e67c6931245599d2a51e2fc3bcd02"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "97043496c577377f9c41d0af2106fa4c"
server_key_iv: "eafce47806852d5ce097518427127d9a"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.399232 10330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance:
uuid: "3b7e67c6931245599d2a51e2fc3bcd02"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "97043496c577377f9c41d0af2106fa4c"
server_key_iv: "eafce47806852d5ce097518427127d9a"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.402841 10330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 04:38:37.405175 10352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.406427 10330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:37.406550 10330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal
uuid: "3b7e67c6931245599d2a51e2fc3bcd02"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "97043496c577377f9c41d0af2106fa4c"
server_key_iv: "eafce47806852d5ce097518427127d9a"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.406644 10330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:37.429215 10330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:37.429890 10330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:37.430064 10330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:37.430652 10330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:37.431584 10330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:37.431632 10330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.431685 10330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:37.431715 10330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.440320 10330 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.193:36391
I20260806 04:38:37.440371 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.193:36391 every 8 connection(s)
I20260806 04:38:37.441211 10330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/data/info.pb
I20260806 04:38:37.446481 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 10330
I20260806 04:38:37.446604 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-0/wal/instance
I20260806 04:38:37.446868 12351 external_mini_cluster.cc:1428] Setting key bd2e1ebcef5d1d55b66bfa850b2cd066
I20260806 04:38:37.448752 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.194:0
--local_ip_for_outbound_sockets=127.12.15.194
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:32997
--builtin_ntp_servers=127.12.15.212:40953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 04:38:37.450301 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.442982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.193:60863 (local address 127.12.15.254:32997)
0806 04:38:37.443105 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.443109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.444113 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 04:38:37.445499 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.445502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.445561 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.445645 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:37.446538 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.447160 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.448487 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.448703 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.448794 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.449847 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:37.449850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:37.449852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:37.449864 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:37.449881 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:37.449889 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:37.449947 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.450092 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.450160 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:38:37.451174 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.443324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:32997 (local address 127.12.15.193:60863)
0806 04:38:37.443966 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:37.444006 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:37.445282 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:37.445830 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:37.445836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:37.445918 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:37.446388 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.446400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.447290 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.447293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.448372 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.448384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.448845 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.448850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.449027 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.449585 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:37.449587 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:37.449626 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:37.449699 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:37.449957 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:37.449967 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:37.449974 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 04:38:37.450080 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 04:38:37.450376 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260806 04:38:37.452409 10518 heartbeater.cc:347] Connected to a master server at 127.12.15.254:32997
I20260806 04:38:37.452627 10518 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:37.453137 10518 heartbeater.cc:511] Master 127.12.15.254:32997 requested a full tablet report, sending...
I20260806 04:38:37.454543 10234 ts_manager.cc:194] Registered new tserver with Master: 3b7e67c6931245599d2a51e2fc3bcd02 (127.12.15.193:36391)
I20260806 04:38:37.455674 10234 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.193:60863
W20260806 04:38:37.559075 10524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:37.559348 10524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:37.559401 10524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:37.563365 10524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:37.563419 10524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:37.563493 10524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.194
I20260806 04:38:37.568199 10524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.194
--webserver_port=0
--tserver_master_addrs=127.12.15.254:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.194
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:37.569303 10524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:37.570760 10524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:37.580227 10532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:37.580250 10529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:37.580353 10524 server_base.cc:1034] running on GCE node
W20260806 04:38:37.580227 10530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:37.580912 10524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:37.581493 10524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:37.582652 10524 hybrid_clock.cc:648] HybridClock initialized: now 1785991117582629 us; error 32 us; skew 500 ppm
I20260806 04:38:37.584446 10524 webserver.cc:460] Webserver started at http://127.12.15.194:37203/ using document root <none> and password file <none>
I20260806 04:38:37.584991 10524 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:37.585052 10524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:37.585253 10524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:37.587282 10524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data/instance:
uuid: "204295db961342c89ada8baa63999031"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "08214646f36367c7228cfb1a7a67cd60"
server_key_iv: "e69c93047af4147ec2ad7f949d4c6be7"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.587836 10524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance:
uuid: "204295db961342c89ada8baa63999031"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "08214646f36367c7228cfb1a7a67cd60"
server_key_iv: "e69c93047af4147ec2ad7f949d4c6be7"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.591488 10524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 04:38:37.593829 10540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.595017 10524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:37.595134 10524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal
uuid: "204295db961342c89ada8baa63999031"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "08214646f36367c7228cfb1a7a67cd60"
server_key_iv: "e69c93047af4147ec2ad7f949d4c6be7"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.595227 10524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:37.611040 10524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:37.611719 10524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:37.611891 10524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:37.612505 10524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:37.613443 10524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:37.613490 10524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.613543 10524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:37.613570 10524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.622066 10524 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.194:34811
I20260806 04:38:37.622098 10706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.194:34811 every 8 connection(s)
I20260806 04:38:37.622939 10524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/data/info.pb
I20260806 04:38:37.625895 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 10524
I20260806 04:38:37.626058 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal/instance
I20260806 04:38:37.626401 12351 external_mini_cluster.cc:1428] Setting key 220b6c6cd9494ded08a6d130504de74a
I20260806 04:38:37.628468 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.195:0
--local_ip_for_outbound_sockets=127.12.15.195
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:32997
--builtin_ntp_servers=127.12.15.212:40953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 04:38:37.632786 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.624687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.194:49125 (local address 127.12.15.254:32997)
0806 04:38:37.624835 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.624839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.626031 (+ 1192us) server_negotiation.cc:407] Connection header received
0806 04:38:37.627645 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.627649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.627704 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.627783 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:37.628745 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.629442 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.630876 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.631088 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.631170 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.632326 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:37.632329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:37.632346 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:37.632361 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:37.632385 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:37.632396 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:37.632450 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.632620 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.632671 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 04:38:37.633900 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.625084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:32997 (local address 127.12.15.194:49125)
0806 04:38:37.625822 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:37.625879 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:37.627420 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:37.627971 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:37.627977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:37.628062 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:37.628601 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.628619 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.629608 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.629611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.630746 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.630759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.631233 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.631238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.631442 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.632038 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:37.632040 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:37.632092 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:37.632169 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:37.632466 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:37.632475 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:37.632481 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 04:38:37.632606 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 04:38:37.632879 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260806 04:38:37.634991 10707 heartbeater.cc:347] Connected to a master server at 127.12.15.254:32997
I20260806 04:38:37.635244 10707 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:37.635878 10707 heartbeater.cc:511] Master 127.12.15.254:32997 requested a full tablet report, sending...
I20260806 04:38:37.637007 10234 ts_manager.cc:194] Registered new tserver with Master: 204295db961342c89ada8baa63999031 (127.12.15.194:34811)
I20260806 04:38:37.637552 10234 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.194:49125
W20260806 04:38:37.742506 10712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:37.742753 10712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:37.742803 10712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:37.746686 10712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:37.746733 10712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:37.746804 10712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.195
I20260806 04:38:37.751341 10712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.195
--webserver_port=0
--tserver_master_addrs=127.12.15.254:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.195
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:37.752331 10712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:37.753669 10712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:37.762941 10725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:37.762974 10723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:37.763095 10712 server_base.cc:1034] running on GCE node
W20260806 04:38:37.762979 10727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:37.763553 10712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:37.764155 10712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:37.765311 10712 hybrid_clock.cc:648] HybridClock initialized: now 1785991117765280 us; error 42 us; skew 500 ppm
I20260806 04:38:37.767022 10712 webserver.cc:460] Webserver started at http://127.12.15.195:33465/ using document root <none> and password file <none>
I20260806 04:38:37.767585 10712 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:37.767652 10712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:37.767846 10712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:37.769914 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data/instance:
uuid: "572534c0544f49a9b626fe3bf24141e7"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "974b1448f02e5f0dce948556e658ddee"
server_key_iv: "67a3ce641b936debc278d886de256fa0"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.770485 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance:
uuid: "572534c0544f49a9b626fe3bf24141e7"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "974b1448f02e5f0dce948556e658ddee"
server_key_iv: "67a3ce641b936debc278d886de256fa0"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.774154 10712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 04:38:37.776538 10738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.777782 10712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:37.777889 10712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal
uuid: "572534c0544f49a9b626fe3bf24141e7"
format_stamp: "Formatted at 2026-08-06 04:38:37 on dist-test-slave-xt4k"
server_key: "974b1448f02e5f0dce948556e658ddee"
server_key_iv: "67a3ce641b936debc278d886de256fa0"
server_key_version: "encryptionkey@0"
I20260806 04:38:37.777983 10712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:37.792564 10712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:37.793184 10712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:37.793344 10712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:37.793896 10712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 04:38:37.794812 10712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 04:38:37.794862 10712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.794914 10712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 04:38:37.794942 10712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:37.803478 10712 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.195:39581
I20260806 04:38:37.803495 10901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.195:39581 every 8 connection(s)
I20260806 04:38:37.804407 10712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/data/info.pb
I20260806 04:38:37.805696 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 10712
I20260806 04:38:37.805819 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal/instance
I20260806 04:38:37.806140 12351 external_mini_cluster.cc:1428] Setting key bd613e62da047527e4beaf7ccc72f7c4
I20260806 04:38:37.813777 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.806342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:42645 (local address 127.12.15.254:32997)
0806 04:38:37.806468 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.806472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.807700 (+ 1228us) server_negotiation.cc:407] Connection header received
0806 04:38:37.809092 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.809103 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.809152 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.809235 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:37.810207 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.810814 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.812011 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.812250 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.812369 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.813397 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:37.813401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:37.813403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:37.813415 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:37.813434 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:37.813442 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:37.813499 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.813613 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.813664 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:38:37.814625 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.806785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:32997 (local address 127.12.15.195:42645)
0806 04:38:37.807548 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:37.807600 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:37.808902 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:37.809444 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:37.809451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:37.809536 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:38:37.810070 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.810084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.810940 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.810943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.811851 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.811862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.812451 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.812455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.812619 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.813148 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 04:38:37.813149 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:38:37.813188 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 04:38:37.813260 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:38:37.813507 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:38:37.813512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:38:37.813516 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 04:38:37.813603 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 04:38:37.813817 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":150,"threads_started":1}
I20260806 04:38:37.815773 10902 heartbeater.cc:347] Connected to a master server at 127.12.15.254:32997
I20260806 04:38:37.816006 10902 heartbeater.cc:464] Registering TS with master...
I20260806 04:38:37.816542 10902 heartbeater.cc:511] Master 127.12.15.254:32997 requested a full tablet report, sending...
I20260806 04:38:37.817528 10234 ts_manager.cc:194] Registered new tserver with Master: 572534c0544f49a9b626fe3bf24141e7 (127.12.15.195:39581)
I20260806 04:38:37.818086 10234 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.195:42645
I20260806 04:38:37.819887 12351 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 04:38:37.825515 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.821865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.12.15.254:32997)
0806 04:38:37.822029 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.822033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.822157 (+ 124us) server_negotiation.cc:407] Connection header received
0806 04:38:37.822280 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.822284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.822328 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.822392 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:37.822696 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.823208 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.824020 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.824233 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.824331 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.825138 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:37.825142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:37.825143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:37.825155 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:37.825173 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:37.825181 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:37.825239 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.825324 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.825377 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 04:38:37.828277 10234 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 04:38:37.831279 10234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 04:38:37.845341 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.839423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36391 (local address 127.0.0.1:56008)
0806 04:38:37.840102 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:37.840150 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:37.840277 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:37.840995 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:37.841002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:37.841027 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:37.841165 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.841171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.842708 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.842711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.843975 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.843987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.844800 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.844803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.844909 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.844947 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 04:38:37.845060 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 04:38:37.845187 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":41,"thread_start_us":75,"threads_started":1}
I20260806 04:38:37.845345 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.839569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.12.15.193:36391)
0806 04:38:37.840469 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.840482 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.840516 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:38:37.840597 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.840603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.840816 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.841298 (+ 482us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:37.841441 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.842594 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.844106 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.844687 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.844789 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.845009 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.845092 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.845191 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":140,"threads_started":1}
I20260806 04:38:37.845631 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.839793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.195:39581 (local address 127.0.0.1:38356)
0806 04:38:37.840642 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:37.840653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:37.840722 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:37.841525 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:37.841528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:37.841547 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:37.841651 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.841658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.842936 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.842940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.843988 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.843999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.845236 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.845240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.845325 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.845348 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 04:38:37.845414 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 04:38:37.845493 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":41,"thread_start_us":95,"threads_started":1}
I20260806 04:38:37.846108 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.840217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.12.15.195:39581)
0806 04:38:37.840946 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.840959 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.840982 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:38:37.841045 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.841050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.841223 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.841370 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:37.841771 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.842838 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.844147 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.844865 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.844968 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.845167 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.845841 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.845944 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":133,"threads_started":1}
I20260806 04:38:37.846477 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.839586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34811 (local address 127.0.0.1:48380)
0806 04:38:37.840498 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:37.840513 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:37.840610 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:37.841359 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:37.841362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:37.841378 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:37.841471 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.841478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.842482 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.842485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.843975 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:37.843987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.846102 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.846106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:37.846195 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.846221 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 04:38:37.846286 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 04:38:37.846357 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":79,"thread_start_us":121,"threads_started":1}
I20260806 04:38:37.848693 10922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:37.840198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.12.15.194:34811)
0806 04:38:37.840836 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:37.840850 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:37.840876 (+ 26us) server_negotiation.cc:407] Connection header received
0806 04:38:37.841038 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:37.841044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:37.841222 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:37.841370 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:37.841584 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.842365 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.844420 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:37.845167 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:37.845266 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:37.846643 (+ 1377us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:37.846694 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 04:38:37.846783 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":137,"threads_started":1}
I20260806 04:38:37.848404 10418 tablet_service.cc:1467] Processing CreateTablet for tablet 732a133786b042c0a29cfef5f60505e0 (DEFAULT_TABLE table=test-table [id=5e5f1ac29402497bbca19b0a306b4aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:37.848464 10804 tablet_service.cc:1467] Processing CreateTablet for tablet 732a133786b042c0a29cfef5f60505e0 (DEFAULT_TABLE table=test-table [id=5e5f1ac29402497bbca19b0a306b4aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:37.849447 10418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 732a133786b042c0a29cfef5f60505e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:37.849489 10804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 732a133786b042c0a29cfef5f60505e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:37.851348 10610 tablet_service.cc:1467] Processing CreateTablet for tablet 732a133786b042c0a29cfef5f60505e0 (DEFAULT_TABLE table=test-table [id=5e5f1ac29402497bbca19b0a306b4aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 04:38:37.852375 10610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 732a133786b042c0a29cfef5f60505e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:37.856031 10925 tablet_bootstrap.cc:492] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Bootstrap starting.
I20260806 04:38:37.857012 10928 tablet_bootstrap.cc:492] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: Bootstrap starting.
I20260806 04:38:37.858107 10925 tablet_bootstrap.cc:654] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:37.858824 10925 log.cc:824] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:37.859084 10928 tablet_bootstrap.cc:654] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:37.859092 10929 tablet_bootstrap.cc:492] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Bootstrap starting.
I20260806 04:38:37.859848 10928 log.cc:824] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:37.860939 10925 tablet_bootstrap.cc:492] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: No bootstrap required, opened a new log
I20260806 04:38:37.861140 10925 ts_tablet_manager.cc:1397] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260806 04:38:37.861202 10929 tablet_bootstrap.cc:654] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:37.861662 10928 tablet_bootstrap.cc:492] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: No bootstrap required, opened a new log
I20260806 04:38:37.861830 10928 ts_tablet_manager.cc:1397] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:38:37.861984 10929 log.cc:824] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:37.863905 10929 tablet_bootstrap.cc:492] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: No bootstrap required, opened a new log
I20260806 04:38:37.864085 10929 ts_tablet_manager.cc:1397] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 04:38:37.865080 10928 raft_consensus.cc:348] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:37.865290 10928 raft_consensus.cc:374] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:37.865352 10928 raft_consensus.cc:729] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7e67c6931245599d2a51e2fc3bcd02, State: Initialized, Role: FOLLOWER
I20260806 04:38:37.865226 10925 raft_consensus.cc:348] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:37.865471 10925 raft_consensus.cc:374] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:37.865526 10925 raft_consensus.cc:729] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572534c0544f49a9b626fe3bf24141e7, State: Initialized, Role: FOLLOWER
I20260806 04:38:37.865818 10928 consensus_queue.cc:260] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:37.865928 10925 consensus_queue.cc:260] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:37.866608 10902 heartbeater.cc:503] Master 127.12.15.254:32997 was elected leader, sending a full tablet report...
I20260806 04:38:37.866866 10925 ts_tablet_manager.cc:1428] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 04:38:37.867260 10928 ts_tablet_manager.cc:1428] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 04:38:37.867439 10518 heartbeater.cc:503] Master 127.12.15.254:32997 was elected leader, sending a full tablet report...
I20260806 04:38:37.868706 10929 raft_consensus.cc:348] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:37.868897 10929 raft_consensus.cc:374] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:37.868952 10929 raft_consensus.cc:729] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 204295db961342c89ada8baa63999031, State: Initialized, Role: FOLLOWER
I20260806 04:38:37.869338 10929 consensus_queue.cc:260] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:37.869928 10707 heartbeater.cc:503] Master 127.12.15.254:32997 was elected leader, sending a full tablet report...
I20260806 04:38:37.870283 10929 ts_tablet_manager.cc:1428] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260806 04:38:37.873776 10710 tablet.cc:2369] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 04:38:37.942370 10519 tablet.cc:2369] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:38.016860 10936 raft_consensus.cc:482] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 04:38:38.017084 10936 raft_consensus.cc:504] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:38.018285 10936 leader_election.cc:288] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b7e67c6931245599d2a51e2fc3bcd02 (127.12.15.193:36391), 204295db961342c89ada8baa63999031 (127.12.15.194:34811)
I20260806 04:38:38.023039 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:38.018652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:50263 (local address 127.12.15.193:36391)
0806 04:38:38.018789 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:38.018793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:38.019175 (+ 382us) server_negotiation.cc:407] Connection header received
0806 04:38:38.019258 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:38.019261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:38.019301 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:38.019358 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:38.019652 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.020365 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.021731 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.022322 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.022460 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:38.022512 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:38.022831 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 04:38:38.022904 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 04:38:38.023039 10922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:38.018661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.195:59165 (local address 127.12.15.194:34811)
0806 04:38:38.018780 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:38.018785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:38.018802 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:38:38.018972 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:38.018975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:38.019027 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:38.019094 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 04:38:38.019414 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.020155 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.021706 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.022322 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.022460 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:38.022512 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:38.022832 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 04:38:38.022904 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:38:38.023097 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:38.018566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.194:34811 (local address 127.12.15.195:59165)
0806 04:38:38.018698 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:38.018714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:38.018834 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:38.019141 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:38.019144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:38.019171 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:38.019300 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:38.019308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.020256 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.020259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:38.021559 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:38.021572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.022476 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.022481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:38.022672 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:38.022717 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 04:38:38.022801 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 04:38:38.022903 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":87,"spinlock_wait_cycles":384}
I20260806 04:38:38.023274 10938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:38.018568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36391 (local address 127.12.15.195:50263)
0806 04:38:38.019065 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:38.019076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 04:38:38.019181 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:38:38.019422 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:38:38.019425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:38:38.019442 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 04:38:38.019547 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:38.019551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.020450 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.020454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:38.021623 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:38:38.021631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.022476 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.022481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:38:38.022650 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:38.022722 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 04:38:38.022796 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 04:38:38.022903 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":195,"spinlock_wait_cycles":7296,"thread_start_us":118,"threads_started":1}
I20260806 04:38:38.023825 10636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a133786b042c0a29cfef5f60505e0" candidate_uuid: "572534c0544f49a9b626fe3bf24141e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204295db961342c89ada8baa63999031" is_pre_election: true
I20260806 04:38:38.023976 10447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a133786b042c0a29cfef5f60505e0" candidate_uuid: "572534c0544f49a9b626fe3bf24141e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" is_pre_election: true
I20260806 04:38:38.024114 10636 raft_consensus.cc:2454] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 572534c0544f49a9b626fe3bf24141e7 in term 0.
I20260806 04:38:38.024276 10447 raft_consensus.cc:2454] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 572534c0544f49a9b626fe3bf24141e7 in term 0.
I20260806 04:38:38.024601 10740 leader_election.cc:302] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 204295db961342c89ada8baa63999031, 572534c0544f49a9b626fe3bf24141e7; no voters:
I20260806 04:38:38.024899 10936 raft_consensus.cc:2781] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 04:38:38.024996 10936 raft_consensus.cc:482] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 04:38:38.025051 10936 raft_consensus.cc:3037] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:38.026124 10936 raft_consensus.cc:504] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:38.026494 10936 leader_election.cc:288] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [CANDIDATE]: Term 1 election: Requested vote from peers 3b7e67c6931245599d2a51e2fc3bcd02 (127.12.15.193:36391), 204295db961342c89ada8baa63999031 (127.12.15.194:34811)
I20260806 04:38:38.026820 10447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a133786b042c0a29cfef5f60505e0" candidate_uuid: "572534c0544f49a9b626fe3bf24141e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7e67c6931245599d2a51e2fc3bcd02"
I20260806 04:38:38.026854 10636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732a133786b042c0a29cfef5f60505e0" candidate_uuid: "572534c0544f49a9b626fe3bf24141e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "204295db961342c89ada8baa63999031"
I20260806 04:38:38.026937 10447 raft_consensus.cc:3037] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:38.026980 10636 raft_consensus.cc:3037] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:38.027935 10447 raft_consensus.cc:2454] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572534c0544f49a9b626fe3bf24141e7 in term 1.
I20260806 04:38:38.027983 10636 raft_consensus.cc:2454] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572534c0544f49a9b626fe3bf24141e7 in term 1.
I20260806 04:38:38.028264 10743 leader_election.cc:302] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7e67c6931245599d2a51e2fc3bcd02, 572534c0544f49a9b626fe3bf24141e7; no voters:
I20260806 04:38:38.028478 10936 raft_consensus.cc:2781] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:38.028694 10936 raft_consensus.cc:686] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 1 LEADER]: Becoming Leader. State: Replica: 572534c0544f49a9b626fe3bf24141e7, State: Running, Role: LEADER
I20260806 04:38:38.029037 10936 consensus_queue.cc:237] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } }
I20260806 04:38:38.032143 10232 catalog_manager.cc:5495] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 572534c0544f49a9b626fe3bf24141e7 (127.12.15.195). New cstate: current_term: 1 leader_uuid: "572534c0544f49a9b626fe3bf24141e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "572534c0544f49a9b626fe3bf24141e7" member_type: VOTER last_known_addr { host: "127.12.15.195" port: 39581 } health_report { overall_health: HEALTHY } } }
W20260806 04:38:38.055383 10903 tablet.cc:2369] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 04:38:38.080999 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:38.077548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38362 (local address 127.12.15.195:39581)
0806 04:38:38.077687 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:38.077692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:38.077706 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:38:38.077785 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:38.077787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:38.077840 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:38.077925 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:38:38.078131 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.078623 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.079479 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:38.079715 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:38.079797 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:38.080068 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:38:38.080611 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:38:38.080725 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:38.080793 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 04:38:38.080855 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 04:38:38.086958 10636 raft_consensus.cc:1261] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 1 FOLLOWER]: Refusing update from remote peer 572534c0544f49a9b626fe3bf24141e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:38.086958 10447 raft_consensus.cc:1261] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 1 FOLLOWER]: Refusing update from remote peer 572534c0544f49a9b626fe3bf24141e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:38:38.087663 10940 consensus_queue.cc:1035] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "204295db961342c89ada8baa63999031" member_type: VOTER last_known_addr { host: "127.12.15.194" port: 34811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:38.087843 10936 consensus_queue.cc:1035] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7e67c6931245599d2a51e2fc3bcd02" member_type: VOTER last_known_addr { host: "127.12.15.193" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:38:38.094409 10944 mvcc.cc:205] Tried to move back new op lower bound from 7315419619676930048 to 7315419619449663488. Current Snapshot: MvccSnapshot[applied={T|T < 7315419619676930048}]
I20260806 04:38:38.094856 10947 mvcc.cc:205] Tried to move back new op lower bound from 7315419619676930048 to 7315419619449663488. Current Snapshot: MvccSnapshot[applied={T|T < 7315419619676930048}]
I20260806 04:38:38.102694 10232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:38.102836 10232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 04:38:38.105202 10232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7960c91a9fb648729591441e1767b218: Sending DeleteTablet for 3 replicas of tablet 732a133786b042c0a29cfef5f60505e0
I20260806 04:38:38.105983 10610 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a133786b042c0a29cfef5f60505e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:38 UTC) from {username='slave'} at 127.0.0.1:48380
I20260806 04:38:38.106010 10418 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a133786b042c0a29cfef5f60505e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:38 UTC) from {username='slave'} at 127.0.0.1:56008
I20260806 04:38:38.106436 10960 tablet_replica.cc:330] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: stopping tablet replica
I20260806 04:38:38.106447 10804 tablet_service.cc:1514] Processing DeleteTablet for tablet 732a133786b042c0a29cfef5f60505e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 04:38:38 UTC) from {username='slave'} at 127.0.0.1:38356
I20260806 04:38:38.106622 10962 tablet_replica.cc:330] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02: stopping tablet replica
I20260806 04:38:38.106890 10960 raft_consensus.cc:2229] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:38.106910 10962 raft_consensus.cc:2229] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 04:38:38.107237 10962 raft_consensus.cc:2258] T 732a133786b042c0a29cfef5f60505e0 P 3b7e67c6931245599d2a51e2fc3bcd02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:38.107251 10960 raft_consensus.cc:2258] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:38.107414 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 10330
I20260806 04:38:38.115418 10966 tablet_replica.cc:330] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: stopping tablet replica
I20260806 04:38:38.115768 10966 raft_consensus.cc:2229] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 04:38:38.116238 10966 raft_consensus.cc:2258] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 04:38:38.117983 10966 ts_tablet_manager.cc:1905] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:38.117993 10960 ts_tablet_manager.cc:1905] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 04:38:38.120800 10966 ts_tablet_manager.cc:1918] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:38.120903 10966 log.cc:1196] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-2/wal/wals/732a133786b042c0a29cfef5f60505e0
I20260806 04:38:38.120857 10960 ts_tablet_manager.cc:1918] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 04:38:38.121155 10960 log.cc:1196] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Deleting WAL directory at /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785990859032933-12351-0/minicluster-data/ts-1/wal/wals/732a133786b042c0a29cfef5f60505e0
I20260806 04:38:38.121213 10966 ts_tablet_manager.cc:1939] T 732a133786b042c0a29cfef5f60505e0 P 572534c0544f49a9b626fe3bf24141e7: Deleting consensus metadata
I20260806 04:38:38.121541 10960 ts_tablet_manager.cc:1939] T 732a133786b042c0a29cfef5f60505e0 P 204295db961342c89ada8baa63999031: Deleting consensus metadata
I20260806 04:38:38.122566 10218 catalog_manager.cc:4841] TS 572534c0544f49a9b626fe3bf24141e7 (127.12.15.195:39581): tablet 732a133786b042c0a29cfef5f60505e0 (table test-table [id=5e5f1ac29402497bbca19b0a306b4aef]) successfully deleted
I20260806 04:38:38.122756 10215 catalog_manager.cc:4841] TS 204295db961342c89ada8baa63999031 (127.12.15.194:34811): tablet 732a133786b042c0a29cfef5f60505e0 (table test-table [id=5e5f1ac29402497bbca19b0a306b4aef]) successfully deleted
W20260806 04:38:38.124050 10217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.193:36391 (error 108)
I20260806 04:38:38.124938 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 10524
I20260806 04:38:38.125142 10921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:38.124822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.193:36391 (local address 127.0.0.1:56010)
0806 04:38:38.124973 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 04:38:38.125053 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.193:36391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 04:38:38.132787 10217 catalog_manager.cc:4584] TS 3b7e67c6931245599d2a51e2fc3bcd02 (127.12.15.193:36391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 732a133786b042c0a29cfef5f60505e0: Network error: Client connection negotiation failed: client connection to 127.12.15.193:36391: connect: Connection refused (error 111)
I20260806 04:38:38.138554 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 10712
I20260806 04:38:38.153249 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 10197
2026-08-06T04:38:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1168 ms)
[----------] 6 tests from EncryptionPolicyTest (12222 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:38 dist-test-slave-xt4k krb5kdc[10971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:38 dist-test-slave-xt4k krb5kdc[10971](info): set up 2 sockets
Aug 06 04:38:38 dist-test-slave-xt4k krb5kdc[10971](info): commencing operation
krb5kdc: starting...
W20260806 04:38:39.299844 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:39 dist-test-slave-xt4k krb5kdc[10971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:39.423954 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:39.537808 10987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:39.538081 10987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:39.538141 10987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:39.542078 10987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:39.542137 10987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:39.542164 10987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:39.542186 10987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:39.542215 10987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:39.547425 10987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:39.548486 10987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:39.549469 10987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:39.557663 10994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:39.557773 10987 server_base.cc:1034] running on GCE node
W20260806 04:38:39.557690 10997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:39.557714 10993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:39.558429 10987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:39.559473 10987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:39.560631 10987 hybrid_clock.cc:648] HybridClock initialized: now 1785991119560614 us; error 34 us; skew 500 ppm
Aug 06 04:38:39 dist-test-slave-xt4k krb5kdc[10971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991119, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:39.563241 10987 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:39.564054 10987 webserver.cc:460] Webserver started at http://127.0.0.1:44359/ using document root <none> and password file <none>
I20260806 04:38:39.564620 10987 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:39.564682 10987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:39.564847 10987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:39.566824 10987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "a848eae12c9d4de390d2fcea86a640ff"
format_stamp: "Formatted at 2026-08-06 04:38:39 on dist-test-slave-xt4k"
server_key: "8e8cda46f0418689a4cb778dcf5486c3"
server_key_iv: "4a04e4fa4df0c591d1e98f1e5a97e17b"
server_key_version: "encryptionkey@0"
I20260806 04:38:39.567337 10987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "a848eae12c9d4de390d2fcea86a640ff"
format_stamp: "Formatted at 2026-08-06 04:38:39 on dist-test-slave-xt4k"
server_key: "8e8cda46f0418689a4cb778dcf5486c3"
server_key_iv: "4a04e4fa4df0c591d1e98f1e5a97e17b"
server_key_version: "encryptionkey@0"
I20260806 04:38:39.570894 10987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 04:38:39.573487 11008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:39.574666 10987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:39.574796 10987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "a848eae12c9d4de390d2fcea86a640ff"
format_stamp: "Formatted at 2026-08-06 04:38:39 on dist-test-slave-xt4k"
server_key: "8e8cda46f0418689a4cb778dcf5486c3"
server_key_iv: "4a04e4fa4df0c591d1e98f1e5a97e17b"
server_key_version: "encryptionkey@0"
I20260806 04:38:39.574887 10987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:39.592733 10987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:39.595556 10987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:39.595736 10987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:39.602644 10987 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38243
I20260806 04:38:39.602669 11093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38243 every 8 connection(s)
I20260806 04:38:39.603592 10987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:39.606456 11094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:39.611008 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 10987
I20260806 04:38:39.611151 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:39.611397 12351 external_mini_cluster.cc:1428] Setting key a4a6f06cda6baca38ee15da7e57eace9
I20260806 04:38:39.612432 11094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff: Bootstrap starting.
I20260806 04:38:39.614786 11094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:39.615631 11094 log.cc:824] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:39.617830 11094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff: No bootstrap required, opened a new log
Aug 06 04:38:39 dist-test-slave-xt4k krb5kdc[10971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 04:38:39.621726 11094 raft_consensus.cc:348] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a848eae12c9d4de390d2fcea86a640ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38243 } }
I20260806 04:38:39.621941 11094 raft_consensus.cc:374] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:39.622020 11094 raft_consensus.cc:729] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a848eae12c9d4de390d2fcea86a640ff, State: Initialized, Role: FOLLOWER
I20260806 04:38:39.622471 11094 consensus_queue.cc:260] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a848eae12c9d4de390d2fcea86a640ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38243 } }
I20260806 04:38:39.622604 11094 raft_consensus.cc:388] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:39.622674 11094 raft_consensus.cc:482] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:39.622768 11094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:39.623587 11094 raft_consensus.cc:504] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a848eae12c9d4de390d2fcea86a640ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38243 } }
I20260806 04:38:39.623906 11094 leader_election.cc:302] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a848eae12c9d4de390d2fcea86a640ff; no voters:
I20260806 04:38:39.624202 11094 leader_election.cc:288] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:39.624368 11105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:39.624605 11105 raft_consensus.cc:686] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [term 1 LEADER]: Becoming Leader. State: Replica: a848eae12c9d4de390d2fcea86a640ff, State: Running, Role: LEADER
I20260806 04:38:39.624943 11105 consensus_queue.cc:237] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a848eae12c9d4de390d2fcea86a640ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38243 } }
I20260806 04:38:39.625550 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:39.612767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59869 (local address 127.0.0.1:38243)
0806 04:38:39.613198 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:39.613214 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:39.613249 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:38:39.613866 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:39.613882 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:39.614212 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:39.614479 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:39.615357 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:39.616409 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:39.617521 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:39.618031 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:39.618157 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:39.620865 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:39.620871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:39.620885 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:39.620912 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:39.622504 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:39.623031 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:39.623037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:39.623041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:39.623150 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:39.623463 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:39.623466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:39.623467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:39.623881 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:39.624066 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:39.624429 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 04:38:39.624683 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":127,"threads_started":1}
I20260806 04:38:39.625556 11094 sys_catalog.cc:564] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:39.627238 11107 sys_catalog.cc:455] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a848eae12c9d4de390d2fcea86a640ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a848eae12c9d4de390d2fcea86a640ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38243 } } }
I20260806 04:38:39.627360 11107 sys_catalog.cc:458] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:39.627487 11108 sys_catalog.cc:455] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a848eae12c9d4de390d2fcea86a640ff. Latest consensus state: current_term: 1 leader_uuid: "a848eae12c9d4de390d2fcea86a640ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a848eae12c9d4de390d2fcea86a640ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38243 } } }
I20260806 04:38:39.627575 11108 sys_catalog.cc:458] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:39.628508 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:39.630453 11116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:39.633543 11116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:39.638505 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:39.630428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59575 (local address 127.0.0.1:38243)
0806 04:38:39.630536 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:39.630540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:39.630771 (+ 231us) server_negotiation.cc:407] Connection header received
0806 04:38:39.630871 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:39.630873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:39.630910 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:39.630972 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:39.631709 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:39.632481 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:39.633512 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:39.633848 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:39.633958 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:39.635567 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:39.635570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:39.635573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:39.635597 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:39.636878 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:39.637325 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:39.637328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:39.637329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:39.637398 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:39.637713 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:39.637716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:39.637718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:39.637881 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:39.638009 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:39.638268 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 04:38:39.638355 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 04:38:39.640219 11116 catalog_manager.cc:1347] Generated new cluster ID: 2279aa869834417187063c9f9fefa841
I20260806 04:38:39.640281 11116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:39.671218 11116 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:39.672187 11116 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:39.679896 11116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a848eae12c9d4de390d2fcea86a640ff: Generated new TSK 0
I20260806 04:38:39.680675 11116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:39.690092 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 10987
W20260806 04:38:39.708863 10969 connection.cc:535] client connection to 127.0.0.1:38243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38243): Connection reset by peer (error 104)
2026-08-06T04:38:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:39 dist-test-slave-xt4k krb5kdc[11145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:39 dist-test-slave-xt4k krb5kdc[11145](info): set up 2 sockets
Aug 06 04:38:39 dist-test-slave-xt4k krb5kdc[11145](info): commencing operation
krb5kdc: starting...
W20260806 04:38:40.843070 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:40 dist-test-slave-xt4k krb5kdc[11145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:40.965265 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:41.076300 11162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:41.076573 11162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:41.076614 11162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:41.080591 11162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:41.080653 11162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:41.080670 11162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:41.080685 11162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:41.080699 11162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:41.085955 11162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:41.086997 11162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:41.087956 11162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:41.096839 11175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:41.096871 11171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:41.097087 11170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:41.097169 11162 server_base.cc:1034] running on GCE node
I20260806 04:38:41.097788 11162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:41.098894 11162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:41.100086 11162 hybrid_clock.cc:648] HybridClock initialized: now 1785991121100064 us; error 36 us; skew 500 ppm
Aug 06 04:38:41 dist-test-slave-xt4k krb5kdc[11145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991121, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:41.102764 11162 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:41.103645 11162 webserver.cc:460] Webserver started at http://127.0.0.1:42459/ using document root <none> and password file <none>
I20260806 04:38:41.104180 11162 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:41.104245 11162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:41.104485 11162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:41.106561 11162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "993168c5fe5d4fa7926bfbac1ef950d2"
format_stamp: "Formatted at 2026-08-06 04:38:41 on dist-test-slave-xt4k"
server_key: "464fc4620a352dac958e420ea6b1ed5f"
server_key_iv: "0357584c72c3aba284ac4a514bf9d01a"
server_key_version: "encryptionkey@0"
I20260806 04:38:41.107105 11162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "993168c5fe5d4fa7926bfbac1ef950d2"
format_stamp: "Formatted at 2026-08-06 04:38:41 on dist-test-slave-xt4k"
server_key: "464fc4620a352dac958e420ea6b1ed5f"
server_key_iv: "0357584c72c3aba284ac4a514bf9d01a"
server_key_version: "encryptionkey@0"
I20260806 04:38:41.110829 11162 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 04:38:41.113282 11186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:41.114521 11162 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:41.114642 11162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "993168c5fe5d4fa7926bfbac1ef950d2"
format_stamp: "Formatted at 2026-08-06 04:38:41 on dist-test-slave-xt4k"
server_key: "464fc4620a352dac958e420ea6b1ed5f"
server_key_iv: "0357584c72c3aba284ac4a514bf9d01a"
server_key_version: "encryptionkey@0"
I20260806 04:38:41.114740 11162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:41.130079 11162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:41.133440 11162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:41.133630 11162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:41.140630 11162 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42053
I20260806 04:38:41.140647 11263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42053 every 8 connection(s)
I20260806 04:38:41.141593 11162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:41.144542 11265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:41.150293 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2: Bootstrap starting.
I20260806 04:38:41.151458 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 11162
I20260806 04:38:41.151609 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:41.151904 12351 external_mini_cluster.cc:1428] Setting key 6c65ee48201f0786bfa468248c9bc775
I20260806 04:38:41.153342 11265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:41.154121 11265 log.cc:824] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:41.156466 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2: No bootstrap required, opened a new log
I20260806 04:38:41.160634 11265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42053 } }
I20260806 04:38:41.160931 11265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:41.161026 11265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993168c5fe5d4fa7926bfbac1ef950d2, State: Initialized, Role: FOLLOWER
I20260806 04:38:41.161417 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:41.153054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57199 (local address 127.0.0.1:42053)
0806 04:38:41.153514 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:41.153530 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:41.153569 (+ 39us) server_negotiation.cc:407] Connection header received
0806 04:38:41.154326 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:41.154347 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:41.154785 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:41.155112 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:41.155629 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:41.156664 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:41.157826 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:41.158322 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:41.158450 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:41.159381 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:41.159386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:41.159399 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:41.159842 (+ 443us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:41.159900 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:41.159924 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:41.160029 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:41.160209 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 04:38:41.160376 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":133,"threads_started":1}
I20260806 04:38:41.161587 11265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42053 } }
I20260806 04:38:41.161754 11265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:41.161828 11265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:41.161934 11265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:41.162856 11265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42053 } }
I20260806 04:38:41.163190 11265 leader_election.cc:302] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993168c5fe5d4fa7926bfbac1ef950d2; no voters:
I20260806 04:38:41.163497 11265 leader_election.cc:288] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:41.163609 11271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:41.163827 11271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [term 1 LEADER]: Becoming Leader. State: Replica: 993168c5fe5d4fa7926bfbac1ef950d2, State: Running, Role: LEADER
I20260806 04:38:41.164140 11271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42053 } }
I20260806 04:38:41.164397 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:41.164728 11265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:41.165907 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42053 } } }
I20260806 04:38:41.166046 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:41.166414 11275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993168c5fe5d4fa7926bfbac1ef950d2. Latest consensus state: current_term: 1 leader_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993168c5fe5d4fa7926bfbac1ef950d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42053 } } }
I20260806 04:38:41.166527 11275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:41.167162 11297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:41.170154 11297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:41.171597 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:41.166846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54273 (local address 127.0.0.1:42053)
0806 04:38:41.166955 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:41.166960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:41.167220 (+ 260us) server_negotiation.cc:407] Connection header received
0806 04:38:41.167361 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:41.167364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:41.167411 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:41.167474 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:41.167890 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:41.168659 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:41.169802 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:41.170117 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:41.170227 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:41.171115 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:41.171119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:41.171122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:41.171135 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:41.171163 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:41.171175 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:41.171257 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:41.171416 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 04:38:41.171451 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:38:41.176276 11297 catalog_manager.cc:1347] Generated new cluster ID: dc5a895b10c74a5bb47d2ea573ce99d7
I20260806 04:38:41.176355 11297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:41.192127 11297 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:41.193145 11297 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:41.198589 11297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 993168c5fe5d4fa7926bfbac1ef950d2: Generated new TSK 0
I20260806 04:38:41.199235 11297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:41.207641 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 11162
W20260806 04:38:41.223071 11143 connection.cc:535] client connection to 127.0.0.1:42053 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42053): Connection reset by peer (error 104)
2026-08-06T04:38:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:41 dist-test-slave-xt4k krb5kdc[11314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:41 dist-test-slave-xt4k krb5kdc[11314](info): set up 2 sockets
Aug 06 04:38:41 dist-test-slave-xt4k krb5kdc[11314](info): commencing operation
krb5kdc: starting...
W20260806 04:38:42.348083 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:42 dist-test-slave-xt4k krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:42.477581 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34939
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:42.588102 11334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:42.588383 11334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:42.588447 11334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:42.592366 11334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:42.592417 11334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:42.592443 11334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:42.592466 11334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:42.592496 11334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:42.597872 11334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34939
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:42.598919 11334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:42.599911 11334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:42.608227 11341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:42.608223 11342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:42.608327 11334 server_base.cc:1034] running on GCE node
W20260806 04:38:42.608217 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:42.609002 11334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:42.610033 11334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:42.611203 11334 hybrid_clock.cc:648] HybridClock initialized: now 1785991122611178 us; error 36 us; skew 500 ppm
Aug 06 04:38:42 dist-test-slave-xt4k krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:42.616289 11334 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:42.617184 11334 webserver.cc:460] Webserver started at http://127.0.0.1:41763/ using document root <none> and password file <none>
I20260806 04:38:42.617724 11334 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:42.617784 11334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:42.617946 11334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:42.619936 11334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "ef919c87cb234b62b54299258cfd3fb0"
format_stamp: "Formatted at 2026-08-06 04:38:42 on dist-test-slave-xt4k"
server_key: "67bfaeb251ae9e76194b290ab27f98f7"
server_key_iv: "11196b77a150cfc2ad700f5accc42bf1"
server_key_version: "encryptionkey@0"
I20260806 04:38:42.620497 11334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "ef919c87cb234b62b54299258cfd3fb0"
format_stamp: "Formatted at 2026-08-06 04:38:42 on dist-test-slave-xt4k"
server_key: "67bfaeb251ae9e76194b290ab27f98f7"
server_key_iv: "11196b77a150cfc2ad700f5accc42bf1"
server_key_version: "encryptionkey@0"
I20260806 04:38:42.624145 11334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 04:38:42.626508 11357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:42.627739 11334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:42.627856 11334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "ef919c87cb234b62b54299258cfd3fb0"
format_stamp: "Formatted at 2026-08-06 04:38:42 on dist-test-slave-xt4k"
server_key: "67bfaeb251ae9e76194b290ab27f98f7"
server_key_iv: "11196b77a150cfc2ad700f5accc42bf1"
server_key_version: "encryptionkey@0"
I20260806 04:38:42.627956 11334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:42.632375 11334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:42.635149 11334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:42.635299 11334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:42.641986 11334 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34939
I20260806 04:38:42.641999 11427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34939 every 8 connection(s)
I20260806 04:38:42.642863 11334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:42.643558 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 11334
I20260806 04:38:42.643692 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:42.643940 12351 external_mini_cluster.cc:1428] Setting key 4d9584987b84b45c336103209855b2dd
I20260806 04:38:42.645689 11431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:42.648849 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:42.645147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46767 (local address 127.0.0.1:34939)
0806 04:38:42.645534 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:42.645553 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:42.645739 (+ 186us) server_negotiation.cc:407] Connection header received
0806 04:38:42.646446 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:42.646462 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:42.646726 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:42.647032 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:42.647698 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:42.647704 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:42.647720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:42.647729 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:42.647780 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:42.647798 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:42.647921 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:42.647997 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 04:38:42.648143 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260806 04:38:42.651432 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0: Bootstrap starting.
I20260806 04:38:42.651593 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:42.653623 11431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:42.654475 11431 log.cc:824] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:42.655090 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:42.653548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57665 (local address 127.0.0.1:34939)
0806 04:38:42.653654 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:42.653658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:42.653774 (+ 116us) server_negotiation.cc:407] Connection header received
0806 04:38:42.653898 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:42.653901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:42.653931 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:42.653992 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:42.654683 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:42.654687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:42.654689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:42.654699 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:42.654729 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:42.654740 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:42.654810 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:42.654926 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 04:38:42.654963 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:38:42.656580 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0: No bootstrap required, opened a new log
I20260806 04:38:42.660015 11431 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef919c87cb234b62b54299258cfd3fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34939 } }
I20260806 04:38:42.660228 11431 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:42.660264 11431 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef919c87cb234b62b54299258cfd3fb0, State: Initialized, Role: FOLLOWER
I20260806 04:38:42.660692 11431 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef919c87cb234b62b54299258cfd3fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34939 } }
I20260806 04:38:42.660816 11431 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:42.660873 11431 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:42.660969 11431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:42.661810 11431 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef919c87cb234b62b54299258cfd3fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34939 } }
I20260806 04:38:42.662125 11431 leader_election.cc:302] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef919c87cb234b62b54299258cfd3fb0; no voters:
I20260806 04:38:42.662410 11431 leader_election.cc:288] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:42.662559 11448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:42.662810 11448 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [term 1 LEADER]: Becoming Leader. State: Replica: ef919c87cb234b62b54299258cfd3fb0, State: Running, Role: LEADER
I20260806 04:38:42.663142 11448 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef919c87cb234b62b54299258cfd3fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34939 } }
I20260806 04:38:42.663533 11431 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:42.664719 11453 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef919c87cb234b62b54299258cfd3fb0. Latest consensus state: current_term: 1 leader_uuid: "ef919c87cb234b62b54299258cfd3fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef919c87cb234b62b54299258cfd3fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34939 } } }
I20260806 04:38:42.664845 11453 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:42.665252 11465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:42.665213 11451 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef919c87cb234b62b54299258cfd3fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef919c87cb234b62b54299258cfd3fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34939 } } }
I20260806 04:38:42.665349 11451 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:42.668141 11465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:42.673619 11465 catalog_manager.cc:1347] Generated new cluster ID: 3a9be3dbf3df40e6ab73ae951d886a78
I20260806 04:38:42.673696 11465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:42.696681 11465 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:42.697088 11465 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:42.703707 11465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef919c87cb234b62b54299258cfd3fb0: Generated new TSK 0
I20260806 04:38:42.704442 11465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:42.719046 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 11334
2026-08-06T04:38:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:42 dist-test-slave-xt4k krb5kdc[11490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:42 dist-test-slave-xt4k krb5kdc[11490](info): set up 2 sockets
Aug 06 04:38:42 dist-test-slave-xt4k krb5kdc[11490](info): commencing operation
krb5kdc: starting...
W20260806 04:38:43.864703 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:43 dist-test-slave-xt4k krb5kdc[11490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:43.992264 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:44.112254 11506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:44.112561 11506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:44.112610 11506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:44.116556 11506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:44.116639 11506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:44.116663 11506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:44.116693 11506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:44.116719 11506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:44.122144 11506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:44.123229 11506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:44.124243 11506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:44.132606 11515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:44.132573 11512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:44.132573 11513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:44.132627 11506 server_base.cc:1034] running on GCE node
I20260806 04:38:44.133383 11506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:44.134438 11506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:44.135612 11506 hybrid_clock.cc:648] HybridClock initialized: now 1785991124135586 us; error 37 us; skew 500 ppm
Aug 06 04:38:44 dist-test-slave-xt4k krb5kdc[11490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991124, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:44.138387 11506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:44.139281 11506 webserver.cc:460] Webserver started at http://127.0.0.1:40883/ using document root <none> and password file <none>
I20260806 04:38:44.139846 11506 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:44.139915 11506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:44.140142 11506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:44.142221 11506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "e5e93b91410340fa8c18a1e10e7bc20d"
format_stamp: "Formatted at 2026-08-06 04:38:44 on dist-test-slave-xt4k"
server_key: "6fa3e6646ce2f2dcfe21150f0af823fa"
server_key_iv: "cd8f3bf443628e1c017bd493d1cda3af"
server_key_version: "encryptionkey@0"
I20260806 04:38:44.142781 11506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "e5e93b91410340fa8c18a1e10e7bc20d"
format_stamp: "Formatted at 2026-08-06 04:38:44 on dist-test-slave-xt4k"
server_key: "6fa3e6646ce2f2dcfe21150f0af823fa"
server_key_iv: "cd8f3bf443628e1c017bd493d1cda3af"
server_key_version: "encryptionkey@0"
I20260806 04:38:44.146502 11506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:38:44.148881 11526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:44.150133 11506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:44.150252 11506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "e5e93b91410340fa8c18a1e10e7bc20d"
format_stamp: "Formatted at 2026-08-06 04:38:44 on dist-test-slave-xt4k"
server_key: "6fa3e6646ce2f2dcfe21150f0af823fa"
server_key_iv: "cd8f3bf443628e1c017bd493d1cda3af"
server_key_version: "encryptionkey@0"
I20260806 04:38:44.150346 11506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:44.167856 11506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:44.170931 11506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:44.171131 11506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:44.178524 11506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39487
I20260806 04:38:44.178543 11605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39487 every 8 connection(s)
I20260806 04:38:44.179510 11506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:44.180699 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 11506
I20260806 04:38:44.180852 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:44.181181 12351 external_mini_cluster.cc:1428] Setting key 4589cc4e46c8d8f6d40b3f2520d209d0
I20260806 04:38:44.182868 11607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:44.188830 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d: Bootstrap starting.
Aug 06 04:38:44 dist-test-slave-xt4k krb5kdc[11490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 04:38:44.191309 11607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:44.192211 11607 log.cc:824] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:44.194621 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d: No bootstrap required, opened a new log
I20260806 04:38:44.195516 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:44.182530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56359 (local address 127.0.0.1:39487)
0806 04:38:44.182951 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:44.182969 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:44.183023 (+ 54us) server_negotiation.cc:407] Connection header received
0806 04:38:44.183731 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:44.183750 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:44.184097 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:44.184402 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:44.185250 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:44.186308 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:44.187163 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:44.187628 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:44.187773 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:44.190509 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:44.190515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:44.190529 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:44.190558 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:44.192437 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:44.192931 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:44.192937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:44.192942 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:44.193056 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:44.193322 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:44.193325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:44.193327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:44.193825 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:44.194024 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:44.194276 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 04:38:44.194585 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260806 04:38:44.198372 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:44.198820 11607 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39487 } }
I20260806 04:38:44.199035 11607 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:44.199088 11607 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e93b91410340fa8c18a1e10e7bc20d, State: Initialized, Role: FOLLOWER
I20260806 04:38:44.199543 11607 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39487 } }
I20260806 04:38:44.199712 11607 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:44.199779 11607 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:44.199874 11607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:44.200950 11607 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39487 } }
I20260806 04:38:44.201278 11607 leader_election.cc:302] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5e93b91410340fa8c18a1e10e7bc20d; no voters:
I20260806 04:38:44.201561 11607 leader_election.cc:288] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:44.201668 11623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:44.201838 11623 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [term 1 LEADER]: Becoming Leader. State: Replica: e5e93b91410340fa8c18a1e10e7bc20d, State: Running, Role: LEADER
I20260806 04:38:44.202147 11623 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39487 } }
I20260806 04:38:44.202889 11607 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:44.203378 11623 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5e93b91410340fa8c18a1e10e7bc20d. Latest consensus state: current_term: 1 leader_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39487 } } }
I20260806 04:38:44.203491 11623 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:44.204056 11624 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e93b91410340fa8c18a1e10e7bc20d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39487 } } }
I20260806 04:38:44.204164 11624 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d [sys.catalog]: This master's current role is: LEADER
W20260806 04:38:44.208356 11642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:38:44.208446 11642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:38:44.208515 11630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:44.209353 11630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:44.209470 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:44.200420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49789 (local address 127.0.0.1:39487)
0806 04:38:44.200534 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:44.200538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:44.200646 (+ 108us) server_negotiation.cc:407] Connection header received
0806 04:38:44.200768 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:44.200772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:44.200815 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:44.200878 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:44.201583 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:44.202279 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:44.203204 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:44.204313 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:44.204441 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:44.206035 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:44.206039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:44.206041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:44.206066 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:44.207524 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:44.207982 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:44.207986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:44.207988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:44.208076 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:44.208403 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:44.208406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:44.208408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:44.208626 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:44.208753 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:44.209016 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 04:38:44.209137 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:38:44.215327 11630 catalog_manager.cc:1347] Generated new cluster ID: f0312d05c3e84ce7affabb238ff12e32
I20260806 04:38:44.215416 11630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:44.230072 11630 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:44.231408 11630 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:44.240954 11630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5e93b91410340fa8c18a1e10e7bc20d: Generated new TSK 0
I20260806 04:38:44.241818 11630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:44.246460 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 11506
2026-08-06T04:38:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:44 dist-test-slave-xt4k krb5kdc[11651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:44 dist-test-slave-xt4k krb5kdc[11651](info): set up 2 sockets
Aug 06 04:38:44 dist-test-slave-xt4k krb5kdc[11651](info): commencing operation
krb5kdc: starting...
W20260806 04:38:45.396036 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:45 dist-test-slave-xt4k krb5kdc[11651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:45.519095 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:45.629168 11682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:45.629436 11682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:45.629483 11682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:45.633349 11682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:45.633406 11682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:45.633424 11682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:45.633440 11682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:45.633460 11682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:45.638564 11682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:45.639539 11682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:45.640520 11682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:45.648754 11694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:45.648804 11687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:45.648939 11682 server_base.cc:1034] running on GCE node
W20260806 04:38:45.648867 11689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:45.649672 11682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:45.650785 11682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:45.651968 11682 hybrid_clock.cc:648] HybridClock initialized: now 1785991125651942 us; error 38 us; skew 500 ppm
Aug 06 04:38:45 dist-test-slave-xt4k krb5kdc[11651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991125, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:45.655007 11682 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:45.655961 11682 webserver.cc:460] Webserver started at http://127.0.0.1:45079/ using document root <none> and password file <none>
I20260806 04:38:45.656551 11682 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:45.656621 11682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:45.656819 11682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:45.658879 11682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "20bbd4130b974532aa7d1911bd1e0d56"
format_stamp: "Formatted at 2026-08-06 04:38:45 on dist-test-slave-xt4k"
server_key: "b36e936844ec7b687b54dc0c8dd85657"
server_key_iv: "7ae12f4d3abbaec834f13beccc985611"
server_key_version: "encryptionkey@0"
I20260806 04:38:45.659420 11682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "20bbd4130b974532aa7d1911bd1e0d56"
format_stamp: "Formatted at 2026-08-06 04:38:45 on dist-test-slave-xt4k"
server_key: "b36e936844ec7b687b54dc0c8dd85657"
server_key_iv: "7ae12f4d3abbaec834f13beccc985611"
server_key_version: "encryptionkey@0"
I20260806 04:38:45.663385 11682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 04:38:45.665869 11703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:45.667117 11682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:38:45.667232 11682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "20bbd4130b974532aa7d1911bd1e0d56"
format_stamp: "Formatted at 2026-08-06 04:38:45 on dist-test-slave-xt4k"
server_key: "b36e936844ec7b687b54dc0c8dd85657"
server_key_iv: "7ae12f4d3abbaec834f13beccc985611"
server_key_version: "encryptionkey@0"
I20260806 04:38:45.667328 11682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:45.694087 11682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:45.697068 11682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:45.697244 11682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:45.703980 11682 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42663
I20260806 04:38:45.704010 11785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42663 every 8 connection(s)
I20260806 04:38:45.704927 11682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:45.705492 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 11682
I20260806 04:38:45.705636 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:45.705891 12351 external_mini_cluster.cc:1428] Setting key 9944b9426ec65142517ef626a7f27c7d
I20260806 04:38:45.707975 11786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:45.714210 11786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56: Bootstrap starting.
I20260806 04:38:45.714555 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:45.707268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46687 (local address 127.0.0.1:42663)
0806 04:38:45.707662 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:45.707677 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:45.707711 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:38:45.708373 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:45.708392 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:45.708732 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:45.708996 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:45.709352 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:45.710327 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:45.711540 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:45.712021 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:45.712148 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:45.712981 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:45.712987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:45.712999 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:45.713355 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:45.713405 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:45.713426 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:45.713519 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:45.713653 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 04:38:45.713841 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260806 04:38:45.716631 11786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:45.717342 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:45.717540 11786 log.cc:824] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:45.719646 11786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56: No bootstrap required, opened a new log
I20260806 04:38:45.723192 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:45.719426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40651 (local address 127.0.0.1:42663)
0806 04:38:45.719546 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:45.719551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:45.719791 (+ 240us) server_negotiation.cc:407] Connection header received
0806 04:38:45.719838 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:45.719840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:45.719884 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:45.719972 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:45.720171 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:45.720895 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:45.721692 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:45.722044 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:45.722149 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:45.722816 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:45.722820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:45.722823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:45.722837 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:45.722865 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:45.722877 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:45.722951 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:45.723011 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 04:38:45.723061 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 04:38:45.723987 11786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bbd4130b974532aa7d1911bd1e0d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42663 } }
I20260806 04:38:45.724231 11786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:45.724272 11786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20bbd4130b974532aa7d1911bd1e0d56, State: Initialized, Role: FOLLOWER
I20260806 04:38:45.724669 11786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bbd4130b974532aa7d1911bd1e0d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42663 } }
I20260806 04:38:45.724785 11786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:45.724823 11786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:45.724891 11786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:45.725684 11786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bbd4130b974532aa7d1911bd1e0d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42663 } }
I20260806 04:38:45.725986 11786 leader_election.cc:302] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20bbd4130b974532aa7d1911bd1e0d56; no voters:
I20260806 04:38:45.726233 11786 leader_election.cc:288] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:45.726431 11803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:45.726815 11803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [term 1 LEADER]: Becoming Leader. State: Replica: 20bbd4130b974532aa7d1911bd1e0d56, State: Running, Role: LEADER
I20260806 04:38:45.727128 11803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bbd4130b974532aa7d1911bd1e0d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42663 } }
I20260806 04:38:45.727368 11786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:45.728894 11804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20bbd4130b974532aa7d1911bd1e0d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bbd4130b974532aa7d1911bd1e0d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42663 } } }
I20260806 04:38:45.728930 11806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20bbd4130b974532aa7d1911bd1e0d56. Latest consensus state: current_term: 1 leader_uuid: "20bbd4130b974532aa7d1911bd1e0d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20bbd4130b974532aa7d1911bd1e0d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42663 } } }
I20260806 04:38:45.729027 11804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:45.729027 11806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:45.729409 11820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:45.732172 11820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:45.737619 11820 catalog_manager.cc:1347] Generated new cluster ID: 098c472276bb4ce08eeb48fd07dad051
I20260806 04:38:45.737689 11820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:45.751756 11820 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:45.752780 11820 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:45.762457 11820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20bbd4130b974532aa7d1911bd1e0d56: Generated new TSK 0
I20260806 04:38:45.763134 11820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:45.767490 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 11682
2026-08-06T04:38:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:45 dist-test-slave-xt4k krb5kdc[11833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:45 dist-test-slave-xt4k krb5kdc[11833](info): set up 2 sockets
Aug 06 04:38:45 dist-test-slave-xt4k krb5kdc[11833](info): commencing operation
krb5kdc: starting...
W20260806 04:38:46.909191 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:46 dist-test-slave-xt4k krb5kdc[11833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:47.035573 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46229
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:47.147799 11849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:47.148043 11849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:47.148099 11849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:47.152070 11849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:47.152132 11849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:47.152159 11849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:47.152181 11849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:47.152211 11849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:47.157402 11849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:47.158396 11849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:47.159363 11849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:47.167887 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:47.167896 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:47.168222 11849 server_base.cc:1034] running on GCE node
W20260806 04:38:47.168294 11860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:47.168869 11849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:47.169924 11849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:47.171098 11849 hybrid_clock.cc:648] HybridClock initialized: now 1785991127171073 us; error 39 us; skew 500 ppm
Aug 06 04:38:47 dist-test-slave-xt4k krb5kdc[11833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991127, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:47.173759 11849 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:47.174592 11849 webserver.cc:460] Webserver started at http://127.0.0.1:45765/ using document root <none> and password file <none>
I20260806 04:38:47.175127 11849 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:47.175192 11849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:47.175392 11849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:47.177397 11849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "ae74a34587a040d2b57db5d704548b19"
format_stamp: "Formatted at 2026-08-06 04:38:47 on dist-test-slave-xt4k"
server_key: "245c56b7e684349c678923a1d3cac6da"
server_key_iv: "7a9847ddcec50311fb0c5a08435a1a53"
server_key_version: "encryptionkey@0"
I20260806 04:38:47.177919 11849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "ae74a34587a040d2b57db5d704548b19"
format_stamp: "Formatted at 2026-08-06 04:38:47 on dist-test-slave-xt4k"
server_key: "245c56b7e684349c678923a1d3cac6da"
server_key_iv: "7a9847ddcec50311fb0c5a08435a1a53"
server_key_version: "encryptionkey@0"
I20260806 04:38:47.181414 11849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 04:38:47.183753 11871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:47.184988 11849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:38:47.185111 11849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "ae74a34587a040d2b57db5d704548b19"
format_stamp: "Formatted at 2026-08-06 04:38:47 on dist-test-slave-xt4k"
server_key: "245c56b7e684349c678923a1d3cac6da"
server_key_iv: "7a9847ddcec50311fb0c5a08435a1a53"
server_key_version: "encryptionkey@0"
I20260806 04:38:47.185201 11849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:47.189505 11849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:47.192229 11849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:47.192421 11849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:47.198874 11849 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46229
I20260806 04:38:47.198886 11948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46229 every 8 connection(s)
I20260806 04:38:47.199740 11849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:47.202075 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 11849
I20260806 04:38:47.202219 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:47.202517 12351 external_mini_cluster.cc:1428] Setting key 0e767c9dccae1eb64da3098bf9e0ecf0
I20260806 04:38:47.202538 11950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:47.207650 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:47.203818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40625 (local address 127.0.0.1:46229)
0806 04:38:47.204238 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:47.204252 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:47.204286 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:38:47.205197 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:47.205217 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:47.205489 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:47.205808 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:47.206523 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:47.206529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:47.206542 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:47.206552 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:47.206604 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:47.206624 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:47.206759 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:47.206838 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 04:38:47.206987 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":127,"threads_started":1}
I20260806 04:38:47.208999 11950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19: Bootstrap starting.
I20260806 04:38:47.210464 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:47.211385 11950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:47.212193 11950 log.cc:824] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:47.214108 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:47.212427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57989 (local address 127.0.0.1:46229)
0806 04:38:47.212563 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:47.212567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:47.212884 (+ 317us) server_negotiation.cc:407] Connection header received
0806 04:38:47.213006 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:47.213008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:47.213047 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:47.213126 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:47.213758 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:47.213762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:47.213765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:47.213777 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:47.213805 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:47.213817 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:47.213904 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:47.213965 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 04:38:47.214000 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:38:47.214533 11950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19: No bootstrap required, opened a new log
I20260806 04:38:47.218091 11950 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae74a34587a040d2b57db5d704548b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46229 } }
I20260806 04:38:47.218333 11950 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:47.218393 11950 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae74a34587a040d2b57db5d704548b19, State: Initialized, Role: FOLLOWER
I20260806 04:38:47.218809 11950 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae74a34587a040d2b57db5d704548b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46229 } }
I20260806 04:38:47.218952 11950 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:47.219019 11950 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:47.219125 11950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:47.220038 11950 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae74a34587a040d2b57db5d704548b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46229 } }
I20260806 04:38:47.220397 11950 leader_election.cc:302] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae74a34587a040d2b57db5d704548b19; no voters:
I20260806 04:38:47.220745 11950 leader_election.cc:288] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:47.220834 11968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:47.221053 11968 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [term 1 LEADER]: Becoming Leader. State: Replica: ae74a34587a040d2b57db5d704548b19, State: Running, Role: LEADER
I20260806 04:38:47.221382 11968 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae74a34587a040d2b57db5d704548b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46229 } }
I20260806 04:38:47.221935 11950 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:47.222913 11970 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae74a34587a040d2b57db5d704548b19. Latest consensus state: current_term: 1 leader_uuid: "ae74a34587a040d2b57db5d704548b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae74a34587a040d2b57db5d704548b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46229 } } }
I20260806 04:38:47.223049 11970 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:47.223434 11969 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae74a34587a040d2b57db5d704548b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae74a34587a040d2b57db5d704548b19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46229 } } }
I20260806 04:38:47.223551 11969 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:47.223434 11983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:47.226759 11983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:47.232301 11983 catalog_manager.cc:1347] Generated new cluster ID: c70b663c46584189ae6a720148c3bf33
I20260806 04:38:47.232437 11983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:47.274071 11983 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:47.274485 11983 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:47.285692 11983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae74a34587a040d2b57db5d704548b19: Generated new TSK 0
I20260806 04:38:47.286641 11983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:47.291649 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 11849
2026-08-06T04:38:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:47 dist-test-slave-xt4k krb5kdc[12000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:47 dist-test-slave-xt4k krb5kdc[12000](info): set up 2 sockets
Aug 06 04:38:47 dist-test-slave-xt4k krb5kdc[12000](info): commencing operation
krb5kdc: starting...
W20260806 04:38:48.426424 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:48 dist-test-slave-xt4k krb5kdc[12000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:48.551492 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:48.665892 12021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:48.666177 12021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:48.666237 12021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:48.670276 12021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:48.670343 12021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:48.670372 12021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:48.670393 12021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:48.670423 12021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:48.675798 12021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:48.676947 12021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:48.677986 12021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:48.686565 12029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:48.686554 12027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:48.686554 12026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:48.686604 12021 server_base.cc:1034] running on GCE node
I20260806 04:38:48.687403 12021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:48.688640 12021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:48.689836 12021 hybrid_clock.cc:648] HybridClock initialized: now 1785991128689807 us; error 38 us; skew 500 ppm
Aug 06 04:38:48 dist-test-slave-xt4k krb5kdc[12000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991128, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:48.695863 12021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:48.696879 12021 webserver.cc:460] Webserver started at http://127.0.0.1:35535/ using document root <none> and password file <none>
I20260806 04:38:48.697448 12021 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:48.697518 12021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:48.697711 12021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:48.699764 12021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "40c85c43aaf54f46bd0e1bf532174ac5"
format_stamp: "Formatted at 2026-08-06 04:38:48 on dist-test-slave-xt4k"
server_key: "1428582c2cfd4851b06e65e9280ece12"
server_key_iv: "e369c215565a7ce3f7c359c9f0f2eee9"
server_key_version: "encryptionkey@0"
I20260806 04:38:48.700306 12021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "40c85c43aaf54f46bd0e1bf532174ac5"
format_stamp: "Formatted at 2026-08-06 04:38:48 on dist-test-slave-xt4k"
server_key: "1428582c2cfd4851b06e65e9280ece12"
server_key_iv: "e369c215565a7ce3f7c359c9f0f2eee9"
server_key_version: "encryptionkey@0"
I20260806 04:38:48.703980 12021 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:38:48.706367 12039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:48.707583 12021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:38:48.707700 12021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "40c85c43aaf54f46bd0e1bf532174ac5"
format_stamp: "Formatted at 2026-08-06 04:38:48 on dist-test-slave-xt4k"
server_key: "1428582c2cfd4851b06e65e9280ece12"
server_key_iv: "e369c215565a7ce3f7c359c9f0f2eee9"
server_key_version: "encryptionkey@0"
I20260806 04:38:48.707790 12021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:48.724459 12021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:48.727486 12021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:48.727665 12021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:48.734449 12021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43833
I20260806 04:38:48.734472 12120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43833 every 8 connection(s)
I20260806 04:38:48.735370 12021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:48.738101 12121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:48.738793 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12021
I20260806 04:38:48.738943 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:48.739265 12351 external_mini_cluster.cc:1428] Setting key 3e02720606d7627b9a444fc30224e438
I20260806 04:38:48.744431 12121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5: Bootstrap starting.
I20260806 04:38:48.747079 12121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:48.747975 12121 log.cc:824] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5: Log is configured to *not* fsync() on all Append() calls
Aug 06 04:38:48 dist-test-slave-xt4k krb5kdc[12000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 04:38:48.750414 12121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5: No bootstrap required, opened a new log
I20260806 04:38:48.754215 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:48.740795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:35531 (local address 127.0.0.1:43833)
0806 04:38:48.741205 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:48.741219 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:48.741254 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:38:48.741878 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:48.741894 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:48.742237 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:48.742521 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:48.743403 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:48.744438 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:48.745554 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:48.746024 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:48.746146 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:48.748984 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:48.748991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:48.749008 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:48.749039 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:48.750991 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:48.751522 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:48.751527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:48.751534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:48.751665 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:48.752007 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:48.752010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:48.752012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:48.752584 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:48.752807 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:48.753010 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 04:38:48.753243 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":102,"threads_started":1}
I20260806 04:38:48.754945 12121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43833 } }
I20260806 04:38:48.755227 12121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:48.755330 12121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c85c43aaf54f46bd0e1bf532174ac5, State: Initialized, Role: FOLLOWER
I20260806 04:38:48.755779 12121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43833 } }
I20260806 04:38:48.755906 12121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:48.755968 12121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:48.756073 12121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:48.757153 12121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43833 } }
I20260806 04:38:48.757280 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:48.757503 12121 leader_election.cc:302] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c85c43aaf54f46bd0e1bf532174ac5; no voters:
I20260806 04:38:48.757809 12121 leader_election.cc:288] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:48.757922 12130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:48.758150 12130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [term 1 LEADER]: Becoming Leader. State: Replica: 40c85c43aaf54f46bd0e1bf532174ac5, State: Running, Role: LEADER
I20260806 04:38:48.758530 12130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43833 } }
I20260806 04:38:48.759083 12121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:48.759814 12133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c85c43aaf54f46bd0e1bf532174ac5. Latest consensus state: current_term: 1 leader_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43833 } } }
I20260806 04:38:48.760008 12133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:48.760215 12131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c85c43aaf54f46bd0e1bf532174ac5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43833 } } }
I20260806 04:38:48.760303 12131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:48.760891 12150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:48.764119 12150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:48.768142 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:48.759419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:53093 (local address 127.0.0.1:43833)
0806 04:38:48.759540 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:48.759544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:48.760522 (+ 978us) server_negotiation.cc:407] Connection header received
0806 04:38:48.760697 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:48.760700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:48.760750 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:48.760818 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:48.761725 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:48.762365 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:48.763162 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:48.763386 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:48.763470 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:48.764908 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:48.764913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:48.764915 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:48.764947 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:48.766269 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:48.766812 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:48.766815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:48.766817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:48.766906 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:48.767274 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:48.767277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:48.767279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:48.767481 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:48.767617 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:48.767898 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 04:38:48.768004 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:38:48.769922 12150 catalog_manager.cc:1347] Generated new cluster ID: 9254f6b1fd1444c7a7257a8ad069cf66
I20260806 04:38:48.770040 12150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:48.780398 12150 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:48.781263 12150 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:48.787355 12150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c85c43aaf54f46bd0e1bf532174ac5: Generated new TSK 0
I20260806 04:38:48.787977 12150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:48.797726 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12021
W20260806 04:38:48.811525 11997 connection.cc:535] client connection to 127.0.0.1:43833 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43833): Connection reset by peer (error 104)
2026-08-06T04:38:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:48 dist-test-slave-xt4k krb5kdc[12166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:48 dist-test-slave-xt4k krb5kdc[12166](info): set up 2 sockets
Aug 06 04:38:48 dist-test-slave-xt4k krb5kdc[12166](info): commencing operation
krb5kdc: starting...
W20260806 04:38:49.955608 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:50 dist-test-slave-xt4k krb5kdc[12166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:50.081835 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:50.203621 12183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:50.203902 12183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:50.203945 12183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:50.208174 12183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:50.208259 12183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:50.208279 12183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:50.208297 12183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:50.208310 12183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:50.214149 12183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:50.215267 12183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:50.216362 12183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:50.225113 12194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:50.225310 12183 server_base.cc:1034] running on GCE node
W20260806 04:38:50.225106 12195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:50.225116 12197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:50.225986 12183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:50.227067 12183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:50.228245 12183 hybrid_clock.cc:648] HybridClock initialized: now 1785991130228230 us; error 27 us; skew 500 ppm
Aug 06 04:38:50 dist-test-slave-xt4k krb5kdc[12166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991130, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:50.230921 12183 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:50.231802 12183 webserver.cc:460] Webserver started at http://127.0.0.1:44049/ using document root <none> and password file <none>
I20260806 04:38:50.232364 12183 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:50.232446 12183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:50.232661 12183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:50.234685 12183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "fc44defbb8f742b282d2503154ca51f4"
format_stamp: "Formatted at 2026-08-06 04:38:50 on dist-test-slave-xt4k"
server_key: "30ca7e97832e10b195cf4d6222ed70c2"
server_key_iv: "5f1164e9a8389e7aa9334718da45febc"
server_key_version: "encryptionkey@0"
I20260806 04:38:50.235220 12183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "fc44defbb8f742b282d2503154ca51f4"
format_stamp: "Formatted at 2026-08-06 04:38:50 on dist-test-slave-xt4k"
server_key: "30ca7e97832e10b195cf4d6222ed70c2"
server_key_iv: "5f1164e9a8389e7aa9334718da45febc"
server_key_version: "encryptionkey@0"
I20260806 04:38:50.238850 12183 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 04:38:50.241202 12207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:50.242355 12183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 04:38:50.242503 12183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "fc44defbb8f742b282d2503154ca51f4"
format_stamp: "Formatted at 2026-08-06 04:38:50 on dist-test-slave-xt4k"
server_key: "30ca7e97832e10b195cf4d6222ed70c2"
server_key_iv: "5f1164e9a8389e7aa9334718da45febc"
server_key_version: "encryptionkey@0"
I20260806 04:38:50.242592 12183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:50.270395 12183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:50.273535 12183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:50.273721 12183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:50.280866 12183 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36967
I20260806 04:38:50.280915 12294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36967 every 8 connection(s)
I20260806 04:38:50.281869 12183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:50.284802 12295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:50.289680 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12183
I20260806 04:38:50.289847 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:50.290158 12351 external_mini_cluster.cc:1428] Setting key 1ae054bda9043a9bbfe5674808c75ae8
I20260806 04:38:50.290779 12295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4: Bootstrap starting.
I20260806 04:38:50.293265 12295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:50.294087 12295 log.cc:824] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:50.296228 12295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4: No bootstrap required, opened a new log
I20260806 04:38:50.299597 12298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:50.291658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:46215 (local address 127.0.0.1:36967)
0806 04:38:50.292139 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:50.292162 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:50.292199 (+ 37us) server_negotiation.cc:407] Connection header received
0806 04:38:50.292864 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:50.292881 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:50.293235 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:50.293527 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:50.293855 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:50.295045 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:50.296204 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:50.296764 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:50.296883 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:50.297813 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:50.297825 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:50.297839 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:50.298230 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:50.298289 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:50.298312 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:50.298418 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:50.298739 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 04:38:50.298873 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":142,"threads_started":1}
I20260806 04:38:50.300089 12295 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44defbb8f742b282d2503154ca51f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260806 04:38:50.300309 12295 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:50.300418 12295 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc44defbb8f742b282d2503154ca51f4, State: Initialized, Role: FOLLOWER
I20260806 04:38:50.300892 12295 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44defbb8f742b282d2503154ca51f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260806 04:38:50.301043 12295 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:50.301113 12295 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:50.301208 12295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:50.302162 12295 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44defbb8f742b282d2503154ca51f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260806 04:38:50.302457 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:50.302520 12295 leader_election.cc:302] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc44defbb8f742b282d2503154ca51f4; no voters:
I20260806 04:38:50.302886 12295 leader_election.cc:288] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:50.302958 12300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:50.303151 12300 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [term 1 LEADER]: Becoming Leader. State: Replica: fc44defbb8f742b282d2503154ca51f4, State: Running, Role: LEADER
I20260806 04:38:50.303491 12300 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44defbb8f742b282d2503154ca51f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260806 04:38:50.304168 12295 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:50.305188 12301 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc44defbb8f742b282d2503154ca51f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44defbb8f742b282d2503154ca51f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } } }
I20260806 04:38:50.305306 12301 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:50.305598 12302 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc44defbb8f742b282d2503154ca51f4. Latest consensus state: current_term: 1 leader_uuid: "fc44defbb8f742b282d2503154ca51f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc44defbb8f742b282d2503154ca51f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } } }
I20260806 04:38:50.305691 12302 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:50.306069 12316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:50.308979 12298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:50.304450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:52447 (local address 127.0.0.1:36967)
0806 04:38:50.304540 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:50.304544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:50.304978 (+ 434us) server_negotiation.cc:407] Connection header received
0806 04:38:50.305163 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:50.305166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:50.305212 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:50.305261 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:50.305654 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:50.306380 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:50.307445 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:50.307664 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:50.307750 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:50.308549 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:50.308553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:50.308556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:50.308568 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:50.308592 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:50.308602 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:50.308665 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:50.308814 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 04:38:50.308846 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 04:38:50.309247 12316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:50.314719 12316 catalog_manager.cc:1347] Generated new cluster ID: 5bb463ed4e4846ff8d5ba1e20eed1f88
I20260806 04:38:50.314797 12316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:50.322078 12316 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:50.323382 12316 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:50.334802 12316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc44defbb8f742b282d2503154ca51f4: Generated new TSK 0
I20260806 04:38:50.335508 12316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:50.340776 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12183
W20260806 04:38:50.356309 12163 connection.cc:535] client connection to 127.0.0.1:36967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36967): Connection reset by peer (error 104)
2026-08-06T04:38:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:50 dist-test-slave-xt4k krb5kdc[12332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:50 dist-test-slave-xt4k krb5kdc[12332](info): set up 2 sockets
Aug 06 04:38:50 dist-test-slave-xt4k krb5kdc[12332](info): commencing operation
krb5kdc: starting...
W20260806 04:38:51.483342 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:51 dist-test-slave-xt4k krb5kdc[12332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:51.605604 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:51.717828 12361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:51.718102 12361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:51.718164 12361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:51.722146 12361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:51.722204 12361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:51.722231 12361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:51.722255 12361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:51.722285 12361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:51.727512 12361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:51.728615 12361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:51.729651 12361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:51.737907 12368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:51.738006 12361 server_base.cc:1034] running on GCE node
W20260806 04:38:51.738031 12371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:51.737926 12369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:51.738732 12361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:51.739868 12361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:51.741035 12361 hybrid_clock.cc:648] HybridClock initialized: now 1785991131741009 us; error 38 us; skew 500 ppm
Aug 06 04:38:51 dist-test-slave-xt4k krb5kdc[12332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991131, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:51.743737 12361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:51.744676 12361 webserver.cc:460] Webserver started at http://127.0.0.1:45789/ using document root <none> and password file <none>
I20260806 04:38:51.745240 12361 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:51.745309 12361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:51.745520 12361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:51.747715 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5"
format_stamp: "Formatted at 2026-08-06 04:38:51 on dist-test-slave-xt4k"
server_key: "06d9774e35d0d826a5ddbfa7b452b87b"
server_key_iv: "21e29c149271be329a7fe5ef8ca3aa89"
server_key_version: "encryptionkey@0"
I20260806 04:38:51.748286 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5"
format_stamp: "Formatted at 2026-08-06 04:38:51 on dist-test-slave-xt4k"
server_key: "06d9774e35d0d826a5ddbfa7b452b87b"
server_key_iv: "21e29c149271be329a7fe5ef8ca3aa89"
server_key_version: "encryptionkey@0"
I20260806 04:38:51.752318 12361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 04:38:51.755008 12380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:51.756367 12361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 04:38:51.756497 12361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5"
format_stamp: "Formatted at 2026-08-06 04:38:51 on dist-test-slave-xt4k"
server_key: "06d9774e35d0d826a5ddbfa7b452b87b"
server_key_iv: "21e29c149271be329a7fe5ef8ca3aa89"
server_key_version: "encryptionkey@0"
I20260806 04:38:51.756599 12361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:51.761597 12361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:51.764770 12361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:51.764946 12361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:51.771809 12361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41367
I20260806 04:38:51.771858 12454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41367 every 8 connection(s)
I20260806 04:38:51.772809 12361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:51.775672 12455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:51.781690 12455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5: Bootstrap starting.
I20260806 04:38:51.781988 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12361
I20260806 04:38:51.782133 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:51.782486 12351 external_mini_cluster.cc:1428] Setting key 2cf35d641ffaf20c8ff7958d9e789251
I20260806 04:38:51.784582 12455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:51.785312 12455 log.cc:824] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:51.787240 12455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5: No bootstrap required, opened a new log
I20260806 04:38:51.787912 12459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:51.783852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:51207 (local address 127.0.0.1:41367)
0806 04:38:51.784328 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:51.784377 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:51.784418 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:38:51.785120 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:51.785139 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:51.785449 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:51.785779 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:51.786694 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:51.786699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:51.786710 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:51.786717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:51.786761 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:51.786777 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:51.786874 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:51.787003 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 04:38:51.787169 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260806 04:38:51.790822 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:51.791322 12455 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41367 } }
I20260806 04:38:51.791543 12455 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:51.791590 12455 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d45a4c5b834dd09c5f1223b3cc77d5, State: Initialized, Role: FOLLOWER
I20260806 04:38:51.792027 12455 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41367 } }
I20260806 04:38:51.792184 12455 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:51.792249 12455 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:51.792364 12455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:51.793463 12455 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41367 } }
I20260806 04:38:51.793794 12455 leader_election.cc:302] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8d45a4c5b834dd09c5f1223b3cc77d5; no voters:
I20260806 04:38:51.794098 12455 leader_election.cc:288] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:51.794174 12469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:51.794374 12469 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [term 1 LEADER]: Becoming Leader. State: Replica: c8d45a4c5b834dd09c5f1223b3cc77d5, State: Running, Role: LEADER
I20260806 04:38:51.794518 12459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:51.792844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:35777 (local address 127.0.0.1:41367)
0806 04:38:51.792963 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:51.792967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:51.793111 (+ 144us) server_negotiation.cc:407] Connection header received
0806 04:38:51.793202 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:51.793205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:51.793243 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:51.793330 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:51.793955 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:51.793959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:51.793962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:51.793973 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:51.794000 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:51.794010 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:51.794080 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:51.794350 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 04:38:51.794398 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:38:51.795403 12455 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:51.794744 12469 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41367 } }
I20260806 04:38:51.797741 12472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8d45a4c5b834dd09c5f1223b3cc77d5. Latest consensus state: current_term: 1 leader_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41367 } } }
I20260806 04:38:51.797873 12471 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d45a4c5b834dd09c5f1223b3cc77d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41367 } } }
I20260806 04:38:51.798172 12472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:51.798172 12471 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5 [sys.catalog]: This master's current role is: LEADER
W20260806 04:38:51.800719 12487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:38:51.800784 12487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:38:51.800854 12489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:51.801653 12489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:51.806871 12489 catalog_manager.cc:1347] Generated new cluster ID: a15b8e4ffb764f618dce84be9557211a
I20260806 04:38:51.806947 12489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:51.820983 12489 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:51.821353 12489 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:51.833036 12489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8d45a4c5b834dd09c5f1223b3cc77d5: Generated new TSK 0
I20260806 04:38:51.833776 12489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:51.839134 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12361
2026-08-06T04:38:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:51 dist-test-slave-xt4k krb5kdc[12496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:51 dist-test-slave-xt4k krb5kdc[12496](info): set up 2 sockets
Aug 06 04:38:51 dist-test-slave-xt4k krb5kdc[12496](info): commencing operation
krb5kdc: starting...
W20260806 04:38:52.990339 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:53 dist-test-slave-xt4k krb5kdc[12496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:53.124037 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:53.243078 12514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:53.243404 12514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:53.243469 12514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:53.247658 12514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:53.247738 12514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:53.247766 12514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:53.247788 12514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:53.247816 12514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:53.253352 12514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:53.254482 12514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:53.255570 12514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:53.264379 12526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:53.264389 12523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:53.264391 12522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:53.264616 12514 server_base.cc:1034] running on GCE node
I20260806 04:38:53.265282 12514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:53.266418 12514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:53.267585 12514 hybrid_clock.cc:648] HybridClock initialized: now 1785991133267562 us; error 33 us; skew 500 ppm
Aug 06 04:38:53 dist-test-slave-xt4k krb5kdc[12496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991133, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:53.270376 12514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:53.271328 12514 webserver.cc:460] Webserver started at http://127.0.0.1:43771/ using document root <none> and password file <none>
I20260806 04:38:53.271880 12514 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:53.271951 12514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:53.272163 12514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:53.274245 12514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "c92322d7d69b43dc9f6fa7962d059f2b"
format_stamp: "Formatted at 2026-08-06 04:38:53 on dist-test-slave-xt4k"
server_key: "1706c2beb99007e410a79997b4141ace"
server_key_iv: "c11d637f1f3cb2e25824eef5e12a3d94"
server_key_version: "encryptionkey@0"
I20260806 04:38:53.274793 12514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "c92322d7d69b43dc9f6fa7962d059f2b"
format_stamp: "Formatted at 2026-08-06 04:38:53 on dist-test-slave-xt4k"
server_key: "1706c2beb99007e410a79997b4141ace"
server_key_iv: "c11d637f1f3cb2e25824eef5e12a3d94"
server_key_version: "encryptionkey@0"
I20260806 04:38:53.278543 12514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:38:53.281020 12538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:53.282308 12514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:53.282457 12514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "c92322d7d69b43dc9f6fa7962d059f2b"
format_stamp: "Formatted at 2026-08-06 04:38:53 on dist-test-slave-xt4k"
server_key: "1706c2beb99007e410a79997b4141ace"
server_key_iv: "c11d637f1f3cb2e25824eef5e12a3d94"
server_key_version: "encryptionkey@0"
I20260806 04:38:53.282558 12514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:53.298399 12514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:53.301522 12514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:53.301699 12514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:53.308415 12514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36709
I20260806 04:38:53.308468 12621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36709 every 8 connection(s)
I20260806 04:38:53.309358 12514 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:53.311357 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12514
I20260806 04:38:53.311489 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:53.311769 12351 external_mini_cluster.cc:1428] Setting key 3d2ce89493ba2dce3a8db3bd9e3e30e4
I20260806 04:38:53.312307 12622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:53.319054 12622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b: Bootstrap starting.
Aug 06 04:38:53 dist-test-slave-xt4k krb5kdc[12496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 04:38:53.321631 12622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:53.322378 12622 log.cc:824] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:53.324361 12622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b: No bootstrap required, opened a new log
I20260806 04:38:53.326116 12630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:53.313049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:44535 (local address 127.0.0.1:36709)
0806 04:38:53.313463 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:53.313478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:53.313513 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:38:53.314275 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:53.314296 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:53.314648 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:53.314924 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:53.315871 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:53.316949 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:53.318095 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:53.318560 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:53.318692 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:53.321543 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:53.321548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:53.321563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:53.321593 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:53.323182 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:53.323697 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:53.323703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:53.323708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:53.323822 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:53.324144 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:53.324146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:53.324147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:53.324611 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:53.324821 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:53.325159 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 04:38:53.325381 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
I20260806 04:38:53.328585 12622 raft_consensus.cc:348] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36709 } }
I20260806 04:38:53.328873 12622 raft_consensus.cc:374] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:53.328960 12622 raft_consensus.cc:729] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c92322d7d69b43dc9f6fa7962d059f2b, State: Initialized, Role: FOLLOWER
I20260806 04:38:53.329500 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:53.329454 12622 consensus_queue.cc:260] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36709 } }
I20260806 04:38:53.329586 12622 raft_consensus.cc:388] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:53.329653 12622 raft_consensus.cc:482] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:53.329751 12622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:53.330693 12622 raft_consensus.cc:504] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36709 } }
I20260806 04:38:53.331041 12622 leader_election.cc:302] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c92322d7d69b43dc9f6fa7962d059f2b; no voters:
I20260806 04:38:53.331329 12622 leader_election.cc:288] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:53.331445 12638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:53.331686 12638 raft_consensus.cc:686] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [term 1 LEADER]: Becoming Leader. State: Replica: c92322d7d69b43dc9f6fa7962d059f2b, State: Running, Role: LEADER
I20260806 04:38:53.332024 12638 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36709 } }
I20260806 04:38:53.332744 12622 sys_catalog.cc:564] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:53.333282 12638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92322d7d69b43dc9f6fa7962d059f2b. Latest consensus state: current_term: 1 leader_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36709 } } }
I20260806 04:38:53.334033 12638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:53.333918 12643 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92322d7d69b43dc9f6fa7962d059f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36709 } } }
I20260806 04:38:53.334349 12643 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:53.335264 12653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:53.338747 12653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:53.340226 12630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:53.331487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:50613 (local address 127.0.0.1:36709)
0806 04:38:53.331595 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:53.331599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:53.331817 (+ 218us) server_negotiation.cc:407] Connection header received
0806 04:38:53.331981 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:53.331985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:53.332030 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:53.332070 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:53.332868 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:53.333551 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:53.334622 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:53.334892 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:53.334999 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:53.336473 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:53.336477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:53.336479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:53.336505 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:53.338043 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:53.338636 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:53.338640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:53.338642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:53.338731 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:53.339135 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:53.339138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:53.339140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:53.339350 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:53.339491 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:53.339800 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 04:38:53.339897 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 04:38:53.344480 12653 catalog_manager.cc:1347] Generated new cluster ID: 89b0f16f87dd465aaebabac032cc9a32
I20260806 04:38:53.344539 12653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:53.360566 12653 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:53.361795 12653 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:53.369690 12653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c92322d7d69b43dc9f6fa7962d059f2b: Generated new TSK 0
I20260806 04:38:53.370529 12653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:53.375082 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12514
W20260806 04:38:53.393567 12494 connection.cc:535] client connection to 127.0.0.1:36709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36709): Connection reset by peer (error 104)
2026-08-06T04:38:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:53 dist-test-slave-xt4k krb5kdc[12671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:53 dist-test-slave-xt4k krb5kdc[12671](info): set up 2 sockets
Aug 06 04:38:53 dist-test-slave-xt4k krb5kdc[12671](info): commencing operation
krb5kdc: starting...
W20260806 04:38:54.524087 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:54 dist-test-slave-xt4k krb5kdc[12671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:54.644279 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:54.759838 12687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:54.760121 12687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:54.760165 12687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:54.764171 12687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:54.764235 12687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:54.764252 12687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:54.764267 12687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:54.764281 12687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:54.769562 12687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37735
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:54.770612 12687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:54.771622 12687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:54.780184 12692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:54.780184 12698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:54.780211 12694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:54.780396 12687 server_base.cc:1034] running on GCE node
I20260806 04:38:54.781148 12687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:54.782284 12687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:54.783456 12687 hybrid_clock.cc:648] HybridClock initialized: now 1785991134783436 us; error 38 us; skew 500 ppm
Aug 06 04:38:54 dist-test-slave-xt4k krb5kdc[12671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:54.790270 12687 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:54.791190 12687 webserver.cc:460] Webserver started at http://127.0.0.1:34539/ using document root <none> and password file <none>
I20260806 04:38:54.791735 12687 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:54.791795 12687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:54.791963 12687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:54.794080 12687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "94572c9bc2884b35b15e11e5168838ea"
format_stamp: "Formatted at 2026-08-06 04:38:54 on dist-test-slave-xt4k"
server_key: "dd8a91642edad0e610093b2f886dd84f"
server_key_iv: "a1518159ea6e8027111ef603eb3646cf"
server_key_version: "encryptionkey@0"
I20260806 04:38:54.794592 12687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "94572c9bc2884b35b15e11e5168838ea"
format_stamp: "Formatted at 2026-08-06 04:38:54 on dist-test-slave-xt4k"
server_key: "dd8a91642edad0e610093b2f886dd84f"
server_key_iv: "a1518159ea6e8027111ef603eb3646cf"
server_key_version: "encryptionkey@0"
I20260806 04:38:54.799297 12687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 04:38:54.806990 12708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:54.808719 12687 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.002s
I20260806 04:38:54.808858 12687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "94572c9bc2884b35b15e11e5168838ea"
format_stamp: "Formatted at 2026-08-06 04:38:54 on dist-test-slave-xt4k"
server_key: "dd8a91642edad0e610093b2f886dd84f"
server_key_iv: "a1518159ea6e8027111ef603eb3646cf"
server_key_version: "encryptionkey@0"
I20260806 04:38:54.808985 12687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:54.826535 12687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:54.831283 12687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:54.831446 12687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:54.838215 12687 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37735
I20260806 04:38:54.838236 12779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37735 every 8 connection(s)
I20260806 04:38:54.839201 12687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:54.841192 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12687
I20260806 04:38:54.841343 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:54.841637 12351 external_mini_cluster.cc:1428] Setting key f7a0bb4e04f0facc3a231105a247f265
I20260806 04:38:54.842358 12780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:54.849329 12780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea: Bootstrap starting.
I20260806 04:38:54.851898 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:54.843063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:46629 (local address 127.0.0.1:37735)
0806 04:38:54.843506 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:54.843525 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:54.843569 (+ 44us) server_negotiation.cc:407] Connection header received
0806 04:38:54.844421 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:54.844446 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:54.844894 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:54.845254 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:54.845680 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:54.847131 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:54.848030 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:54.848749 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:54.848900 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:54.849853 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:54.849860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:54.849875 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:54.850355 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:54.850422 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:54.850456 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:54.850622 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:54.850816 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 04:38:54.850955 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260806 04:38:54.852144 12780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:54.853063 12780 log.cc:824] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:54.855104 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:54.855298 12780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea: No bootstrap required, opened a new log
I20260806 04:38:54.859098 12780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94572c9bc2884b35b15e11e5168838ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37735 } }
I20260806 04:38:54.859349 12780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:54.859385 12780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94572c9bc2884b35b15e11e5168838ea, State: Initialized, Role: FOLLOWER
I20260806 04:38:54.859799 12780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94572c9bc2884b35b15e11e5168838ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37735 } }
I20260806 04:38:54.859972 12780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:54.860036 12780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:54.860137 12780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:54.861349 12780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94572c9bc2884b35b15e11e5168838ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37735 } }
I20260806 04:38:54.861660 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:54.857552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:51701 (local address 127.0.0.1:37735)
0806 04:38:54.857687 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:54.857692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:54.857708 (+ 16us) server_negotiation.cc:407] Connection header received
0806 04:38:54.857745 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:54.857748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:54.857795 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:54.857878 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:54.858182 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:54.858912 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:54.859854 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:54.860180 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:54.860292 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:54.861201 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:54.861205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:54.861208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:54.861222 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:54.861252 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:54.861266 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:54.861367 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:54.861483 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 04:38:54.861525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 04:38:54.861752 12780 leader_election.cc:302] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94572c9bc2884b35b15e11e5168838ea; no voters:
I20260806 04:38:54.862121 12780 leader_election.cc:288] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:54.862207 12795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:54.862438 12795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [term 1 LEADER]: Becoming Leader. State: Replica: 94572c9bc2884b35b15e11e5168838ea, State: Running, Role: LEADER
I20260806 04:38:54.862777 12795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94572c9bc2884b35b15e11e5168838ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37735 } }
I20260806 04:38:54.863474 12780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:54.864544 12797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94572c9bc2884b35b15e11e5168838ea. Latest consensus state: current_term: 1 leader_uuid: "94572c9bc2884b35b15e11e5168838ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94572c9bc2884b35b15e11e5168838ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37735 } } }
I20260806 04:38:54.864544 12796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94572c9bc2884b35b15e11e5168838ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94572c9bc2884b35b15e11e5168838ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37735 } } }
I20260806 04:38:54.864694 12797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:54.864694 12796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:54.865104 12804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:54.867913 12804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:54.873596 12804 catalog_manager.cc:1347] Generated new cluster ID: d35ed04f8f384a4c963594217331aa5c
I20260806 04:38:54.873667 12804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:54.884842 12804 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:54.886125 12804 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:54.894073 12804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94572c9bc2884b35b15e11e5168838ea: Generated new TSK 0
I20260806 04:38:54.894963 12804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:54.901574 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12687
W20260806 04:38:54.917814 12668 connection.cc:535] client connection to 127.0.0.1:37735 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37735): Connection reset by peer (error 104)
2026-08-06T04:38:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:54 dist-test-slave-xt4k krb5kdc[12821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:54 dist-test-slave-xt4k krb5kdc[12821](info): set up 2 sockets
Aug 06 04:38:54 dist-test-slave-xt4k krb5kdc[12821](info): commencing operation
krb5kdc: starting...
W20260806 04:38:56.051651 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:56 dist-test-slave-xt4k krb5kdc[12821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 04:38:56.177177 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:56.290448 12842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:56.290737 12842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:56.290781 12842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:56.294889 12842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:56.294971 12842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:56.294991 12842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:56.295006 12842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:56.295022 12842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:56.300513 12842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:56.301653 12842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:56.302729 12842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:56.311482 12856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:56.311422 12849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:56.311508 12842 server_base.cc:1034] running on GCE node
W20260806 04:38:56.311411 12851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:56.312402 12842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:56.313556 12842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:56.314745 12842 hybrid_clock.cc:648] HybridClock initialized: now 1785991136314722 us; error 39 us; skew 500 ppm
Aug 06 04:38:56 dist-test-slave-xt4k krb5kdc[12821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:56.317842 12842 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 04:38:56.318809 12842 webserver.cc:460] Webserver started at http://127.0.0.1:35349/ using document root <none> and password file <none>
I20260806 04:38:56.319353 12842 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:56.319414 12842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:56.319599 12842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:56.321792 12842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26"
format_stamp: "Formatted at 2026-08-06 04:38:56 on dist-test-slave-xt4k"
server_key: "4a105e7a3d75cd5a1a8f07a1275766c8"
server_key_iv: "7bea729f64bbeb58fb5b8f96d8b3259f"
server_key_version: "encryptionkey@0"
I20260806 04:38:56.322319 12842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26"
format_stamp: "Formatted at 2026-08-06 04:38:56 on dist-test-slave-xt4k"
server_key: "4a105e7a3d75cd5a1a8f07a1275766c8"
server_key_iv: "7bea729f64bbeb58fb5b8f96d8b3259f"
server_key_version: "encryptionkey@0"
I20260806 04:38:56.326200 12842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 04:38:56.328583 12866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:56.330077 12842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:38:56.330204 12842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26"
format_stamp: "Formatted at 2026-08-06 04:38:56 on dist-test-slave-xt4k"
server_key: "4a105e7a3d75cd5a1a8f07a1275766c8"
server_key_iv: "7bea729f64bbeb58fb5b8f96d8b3259f"
server_key_version: "encryptionkey@0"
I20260806 04:38:56.330304 12842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:56.334878 12842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:56.337715 12842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:56.337853 12842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:56.344585 12842 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37369
I20260806 04:38:56.344620 12944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37369 every 8 connection(s)
I20260806 04:38:56.345479 12842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:56.348181 12945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:56.353740 12945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26: Bootstrap starting.
I20260806 04:38:56.353956 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 12842
I20260806 04:38:56.354089 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:56.354336 12351 external_mini_cluster.cc:1428] Setting key 603a7450175fe77030a52d8b0d7d4ce2
I20260806 04:38:56.356266 12945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:56.356990 12945 log.cc:824] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:56.359025 12945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26: No bootstrap required, opened a new log
I20260806 04:38:56.359172 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:56.355593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:35499 (local address 127.0.0.1:37369)
0806 04:38:56.356031 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:56.356045 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:56.356079 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:38:56.356785 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:56.356804 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:56.357075 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:56.357375 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:56.358043 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:56.358048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:56.358059 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:56.358069 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:56.358122 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:56.358141 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:56.358232 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:56.358341 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 04:38:56.358504 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":129,"threads_started":1}
I20260806 04:38:56.361896 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:56.363006 12945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37369 } }
I20260806 04:38:56.363289 12945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:56.363353 12945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d1ac3cc5b64b0789a37e3dd9a7dd26, State: Initialized, Role: FOLLOWER
I20260806 04:38:56.363793 12945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37369 } }
I20260806 04:38:56.363955 12945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:56.364024 12945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:56.364121 12945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:56.365003 12945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37369 } }
I20260806 04:38:56.365307 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:56.363786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:38065 (local address 127.0.0.1:37369)
0806 04:38:56.363903 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:56.363907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:56.364044 (+ 137us) server_negotiation.cc:407] Connection header received
0806 04:38:56.364203 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:56.364206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:56.364241 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:56.364310 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:56.364962 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:56.364966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:56.364968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:56.364979 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:56.365006 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:56.365017 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:56.365085 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:56.365159 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 04:38:56.365197 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:38:56.365355 12945 leader_election.cc:302] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90d1ac3cc5b64b0789a37e3dd9a7dd26; no voters:
I20260806 04:38:56.365679 12945 leader_election.cc:288] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:56.365758 12960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:56.365928 12960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [term 1 LEADER]: Becoming Leader. State: Replica: 90d1ac3cc5b64b0789a37e3dd9a7dd26, State: Running, Role: LEADER
I20260806 04:38:56.366222 12960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37369 } }
I20260806 04:38:56.366937 12945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:56.367798 12962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90d1ac3cc5b64b0789a37e3dd9a7dd26. Latest consensus state: current_term: 1 leader_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37369 } } }
I20260806 04:38:56.368167 12962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:56.368064 12961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d1ac3cc5b64b0789a37e3dd9a7dd26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37369 } } }
I20260806 04:38:56.368502 12961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26 [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:56.369139 12974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:56.372246 12974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:56.377364 12974 catalog_manager.cc:1347] Generated new cluster ID: db3d3f772d874d0784fbb482ef3acf8c
I20260806 04:38:56.377440 12974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:56.389616 12974 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:56.389933 12974 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:56.398993 12974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90d1ac3cc5b64b0789a37e3dd9a7dd26: Generated new TSK 0
I20260806 04:38:56.399662 12974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:56.410383 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 12842
2026-08-06T04:38:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:56 dist-test-slave-xt4k krb5kdc[12987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:56 dist-test-slave-xt4k krb5kdc[12987](info): set up 2 sockets
Aug 06 04:38:56 dist-test-slave-xt4k krb5kdc[12987](info): commencing operation
krb5kdc: starting...
W20260806 04:38:57.552699 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:57 dist-test-slave-xt4k krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:38:57.682035 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38041
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:38041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:57.795147 13007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:57.795403 13007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:57.795470 13007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:57.799438 13007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:57.799502 13007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:57.799530 13007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:57.799551 13007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:57.799575 13007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:57.804749 13007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:38041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:38041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:57.805779 13007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:57.806763 13007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:57.815124 13017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:57.815111 13023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:57.815119 13020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:57.815225 13007 server_base.cc:1034] running on GCE node
I20260806 04:38:57.815894 13007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:57.817000 13007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:57.818162 13007 hybrid_clock.cc:648] HybridClock initialized: now 1785991137818098 us; error 77 us; skew 500 ppm
Aug 06 04:38:57 dist-test-slave-xt4k krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991137, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:57.820827 13007 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:38:57.821717 13007 webserver.cc:460] Webserver started at http://127.12.15.254:35279/ using document root <none> and password file <none>
I20260806 04:38:57.822256 13007 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:57.822322 13007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:57.822541 13007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:57.824687 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "ab328dd8e0e2435baf4b1b1455c9372d"
format_stamp: "Formatted at 2026-08-06 04:38:57 on dist-test-slave-xt4k"
server_key: "8084ab6dc0d9e08a382eac639799362e"
server_key_iv: "d58e216c36265a72b69016308ee14e32"
server_key_version: "encryptionkey@0"
I20260806 04:38:57.825206 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "ab328dd8e0e2435baf4b1b1455c9372d"
format_stamp: "Formatted at 2026-08-06 04:38:57 on dist-test-slave-xt4k"
server_key: "8084ab6dc0d9e08a382eac639799362e"
server_key_iv: "d58e216c36265a72b69016308ee14e32"
server_key_version: "encryptionkey@0"
I20260806 04:38:57.828879 13007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 04:38:57.831169 13034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:57.832360 13007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:38:57.832484 13007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "ab328dd8e0e2435baf4b1b1455c9372d"
format_stamp: "Formatted at 2026-08-06 04:38:57 on dist-test-slave-xt4k"
server_key: "8084ab6dc0d9e08a382eac639799362e"
server_key_iv: "d58e216c36265a72b69016308ee14e32"
server_key_version: "encryptionkey@0"
I20260806 04:38:57.832577 13007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:57.860121 13007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:57.863077 13007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:57.863260 13007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:57.870054 13007 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:38041
I20260806 04:38:57.870067 13104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:38041 every 8 connection(s)
I20260806 04:38:57.871037 13007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:57.873730 13105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:57.879083 13105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d: Bootstrap starting.
I20260806 04:38:57.879724 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13007
I20260806 04:38:57.879880 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:57.880143 12351 external_mini_cluster.cc:1428] Setting key aaae8147eaf3caa012048649bdb31c04
I20260806 04:38:57.881937 13105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:57.882660 13105 log.cc:824] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:57.884872 13105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d: No bootstrap required, opened a new log
Aug 06 04:38:57 dist-test-slave-xt4k krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:38:57.889142 13105 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38041 } }
I20260806 04:38:57.889382 13105 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:57.889441 13105 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab328dd8e0e2435baf4b1b1455c9372d, State: Initialized, Role: FOLLOWER
I20260806 04:38:57.889940 13105 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38041 } }
I20260806 04:38:57.890071 13105 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:57.890126 13105 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:57.890223 13105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:57.891067 13105 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38041 } }
I20260806 04:38:57.891379 13105 leader_election.cc:302] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab328dd8e0e2435baf4b1b1455c9372d; no voters:
I20260806 04:38:57.891664 13105 leader_election.cc:288] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:57.891812 13118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:57.892032 13118 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [term 1 LEADER]: Becoming Leader. State: Replica: ab328dd8e0e2435baf4b1b1455c9372d, State: Running, Role: LEADER
I20260806 04:38:57.892374 13118 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38041 } }
I20260806 04:38:57.893035 13105 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:57.893975 13120 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38041 } } }
I20260806 04:38:57.894109 13120 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:57.894430 13119 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab328dd8e0e2435baf4b1b1455c9372d. Latest consensus state: current_term: 1 leader_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab328dd8e0e2435baf4b1b1455c9372d" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 38041 } } }
I20260806 04:38:57.894538 13119 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:57.895984 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:57.881432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38835 (local address 127.12.15.254:38041)
0806 04:38:57.881872 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:57.881887 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:57.881939 (+ 52us) server_negotiation.cc:407] Connection header received
0806 04:38:57.882614 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:57.882633 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:57.883031 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:57.883408 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:57.884206 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:57.885286 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:57.886424 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:57.886911 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:57.887035 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:57.889933 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:57.889938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:57.889952 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:57.889982 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:57.891762 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:57.892491 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:57.892497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:57.892502 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:57.892638 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:57.893105 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:57.893109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:57.893110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:57.893628 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:57.893852 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:57.894760 (+ 908us) server_negotiation.cc:299] Negotiation successful
0806 04:38:57.895011 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":137,"threads_started":1}
W20260806 04:38:57.898186 13143 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:38:57.898265 13143 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:38:57.898329 13131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:57.898746 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:57.899181 13131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:57.904631 13131 catalog_manager.cc:1347] Generated new cluster ID: a2e78f4cd7d248dd84b05ef3cd42f23e
I20260806 04:38:57.904687 13131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:57.910782 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:57.901440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43971 (local address 127.12.15.254:38041)
0806 04:38:57.901551 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:57.901555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:57.901872 (+ 317us) server_negotiation.cc:407] Connection header received
0806 04:38:57.901986 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:57.901988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:57.902031 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:57.902112 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:57.902905 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:57.903522 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:57.904690 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:57.905039 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:57.905115 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:57.906882 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:57.906886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:57.906888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:38:57.906911 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:57.908410 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:57.908921 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:57.908925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:57.908927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:57.909012 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:38:57.909364 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:38:57.909367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:38:57.909369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:38:57.909563 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:57.909715 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:57.910588 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 04:38:57.910675 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 04:38:57.911465 13131 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:57.912387 13131 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:57.920441 13131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab328dd8e0e2435baf4b1b1455c9372d: Generated new TSK 0
I20260806 04:38:57.921154 13131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:57.925724 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13007
W20260806 04:38:57.934556 12985 connection.cc:535] client connection to 127.12.15.254:38041 recv error: Network error: failed to read from TLS socket (remote: 127.12.15.254:38041): Connection reset by peer (error 104)
2026-08-06T04:38:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:57 dist-test-slave-xt4k krb5kdc[13159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:57 dist-test-slave-xt4k krb5kdc[13159](info): set up 2 sockets
Aug 06 04:38:57 dist-test-slave-xt4k krb5kdc[13159](info): commencing operation
krb5kdc: starting...
W20260806 04:38:59.071223 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:38:59 dist-test-slave-xt4k krb5kdc[13159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:38:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:38:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:38:59.199020 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:45151
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:45151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:38:59.312654 13181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:38:59.312907 13181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:38:59.312947 13181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:38:59.316910 13181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:38:59.316962 13181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:38:59.316979 13181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:38:59.316994 13181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:38:59.317010 13181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:38:59.322283 13181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:45151
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:38:59.323283 13181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:38:59.324262 13181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:38:59.332428 13191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:38:59.332439 13188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:59.332549 13181 server_base.cc:1034] running on GCE node
W20260806 04:38:59.332443 13189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:38:59.333257 13181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:38:59.334267 13181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:38:59.335422 13181 hybrid_clock.cc:648] HybridClock initialized: now 1785991139335398 us; error 35 us; skew 500 ppm
Aug 06 04:38:59 dist-test-slave-xt4k krb5kdc[13159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991139, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:38:59.338037 13181 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:38:59.338868 13181 webserver.cc:460] Webserver started at http://127.12.15.254:33999/ using document root <none> and password file <none>
I20260806 04:38:59.339378 13181 fs_manager.cc:359] Metadata directory not provided
I20260806 04:38:59.339432 13181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:38:59.339591 13181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:38:59.341568 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc"
format_stamp: "Formatted at 2026-08-06 04:38:59 on dist-test-slave-xt4k"
server_key: "06c6bf2450d9ec1b8a3fc0f7ba2f2200"
server_key_iv: "941cdfa39369f263e8864dd674f37b54"
server_key_version: "encryptionkey@0"
I20260806 04:38:59.342065 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc"
format_stamp: "Formatted at 2026-08-06 04:38:59 on dist-test-slave-xt4k"
server_key: "06c6bf2450d9ec1b8a3fc0f7ba2f2200"
server_key_iv: "941cdfa39369f263e8864dd674f37b54"
server_key_version: "encryptionkey@0"
I20260806 04:38:59.345619 13181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:38:59.348104 13202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:38:59.349418 13181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:38:59.349550 13181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc"
format_stamp: "Formatted at 2026-08-06 04:38:59 on dist-test-slave-xt4k"
server_key: "06c6bf2450d9ec1b8a3fc0f7ba2f2200"
server_key_iv: "941cdfa39369f263e8864dd674f37b54"
server_key_version: "encryptionkey@0"
I20260806 04:38:59.349654 13181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:38:59.366622 13181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:38:59.369875 13181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:38:59.370060 13181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:38:59.377067 13181 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:45151
I20260806 04:38:59.377086 13284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:45151 every 8 connection(s)
I20260806 04:38:59.377995 13181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:38:59.380757 13285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:38:59.385877 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13181
I20260806 04:38:59.386015 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:38:59.386265 12351 external_mini_cluster.cc:1428] Setting key 2cec950e7af3c631a015eadd9005082a
I20260806 04:38:59.386336 13285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc: Bootstrap starting.
I20260806 04:38:59.388792 13285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc: Neither blocks nor log segments found. Creating new log.
I20260806 04:38:59.389612 13285 log.cc:824] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc: Log is configured to *not* fsync() on all Append() calls
I20260806 04:38:59.391678 13285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc: No bootstrap required, opened a new log
I20260806 04:38:59.395061 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:59.387562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43237 (local address 127.12.15.254:45151)
0806 04:38:59.387965 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:59.387981 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:59.388017 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:38:59.388685 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:59.388703 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:59.389066 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:59.389391 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:59.389750 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:59.390866 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:59.391991 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:59.392490 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:59.392613 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:59.393436 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:59.393441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:59.393454 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:59.393832 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:59.393886 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:59.393908 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:59.394006 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:59.394185 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 04:38:59.394311 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":109,"threads_started":1}
I20260806 04:38:59.395591 13285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45151 } }
I20260806 04:38:59.395802 13285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:38:59.395851 13285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d79a8802d4a4c37abcfa63a2e4ddbdc, State: Initialized, Role: FOLLOWER
I20260806 04:38:59.396257 13285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45151 } }
I20260806 04:38:59.396406 13285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:38:59.396472 13285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:38:59.396570 13285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:38:59.397531 13285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45151 } }
I20260806 04:38:59.397828 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:38:59.397890 13285 leader_election.cc:302] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d79a8802d4a4c37abcfa63a2e4ddbdc; no voters:
I20260806 04:38:59.398247 13285 leader_election.cc:288] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:38:59.398293 13292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:38:59.398497 13292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [term 1 LEADER]: Becoming Leader. State: Replica: 6d79a8802d4a4c37abcfa63a2e4ddbdc, State: Running, Role: LEADER
I20260806 04:38:59.398854 13292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45151 } }
I20260806 04:38:59.399509 13285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:38:59.400504 13293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45151 } } }
I20260806 04:38:59.400636 13293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:59.400940 13294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d79a8802d4a4c37abcfa63a2e4ddbdc. Latest consensus state: current_term: 1 leader_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a8802d4a4c37abcfa63a2e4ddbdc" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 45151 } } }
I20260806 04:38:59.401019 13294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc [sys.catalog]: This master's current role is: LEADER
I20260806 04:38:59.402107 13311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:38:59.404273 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:38:59.399765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33817 (local address 127.12.15.254:45151)
0806 04:38:59.399867 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 04:38:59.399871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:38:59.400286 (+ 415us) server_negotiation.cc:407] Connection header received
0806 04:38:59.400512 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:38:59.400515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:38:59.400569 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:38:59.400619 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:38:59.401013 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:59.401671 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:59.402682 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:38:59.402908 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:38:59.402995 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:38:59.403776 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:38:59.403780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:38:59.403782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:38:59.403795 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:38:59.403821 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:38:59.403834 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:38:59.403912 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 04:38:59.404111 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 04:38:59.404146 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 04:38:59.405018 13311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:38:59.410974 13311 catalog_manager.cc:1347] Generated new cluster ID: 207f8504e04f49fb8eba477e29e2df7b
I20260806 04:38:59.411037 13311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:38:59.420845 13311 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:38:59.421751 13311 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:38:59.430796 13311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d79a8802d4a4c37abcfa63a2e4ddbdc: Generated new TSK 0
I20260806 04:38:59.431648 13311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:38:59.441841 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13181
W20260806 04:38:59.461839 13154 connection.cc:535] client connection to 127.12.15.254:45151 recv error: Network error: failed to read from TLS socket (remote: 127.12.15.254:45151): Connection reset by peer (error 104)
2026-08-06T04:38:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:38:59 dist-test-slave-xt4k krb5kdc[13331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:38:59 dist-test-slave-xt4k krb5kdc[13331](info): set up 2 sockets
Aug 06 04:38:59 dist-test-slave-xt4k krb5kdc[13331](info): commencing operation
krb5kdc: starting...
W20260806 04:39:00.597688 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:39:00 dist-test-slave-xt4k krb5kdc[13331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:39:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:39:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:39:00.727034 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:35415
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:41337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:35415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:39:00.841565 13352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:00.841822 13352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:00.841877 13352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:00.845841 13352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:00.845903 13352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:00.845930 13352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:00.845950 13352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:00.845980 13352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:00.851138 13352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:41337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:35415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:35415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:00.852160 13352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:00.853160 13352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:00.861377 13360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:00.861377 13359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:00.861387 13362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:00.861588 13352 server_base.cc:1034] running on GCE node
I20260806 04:39:00.862223 13352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:00.863273 13352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:00.864411 13352 hybrid_clock.cc:648] HybridClock initialized: now 1785991140864388 us; error 29 us; skew 500 ppm
Aug 06 04:39:00 dist-test-slave-xt4k krb5kdc[13331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991140, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:00.870146 13352 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:39:00.871063 13352 webserver.cc:460] Webserver started at http://127.12.15.254:44321/ using document root <none> and password file <none>
I20260806 04:39:00.871616 13352 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:00.871681 13352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:00.871881 13352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:00.873945 13352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "88b82878b1c649da8ad6ff229c2bad47"
format_stamp: "Formatted at 2026-08-06 04:39:00 on dist-test-slave-xt4k"
server_key: "420608baa3ccb7e677dac70e1056fb9d"
server_key_iv: "d8701e6ed1eb38b1014517d224f21f0f"
server_key_version: "encryptionkey@0"
I20260806 04:39:00.874481 13352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "88b82878b1c649da8ad6ff229c2bad47"
format_stamp: "Formatted at 2026-08-06 04:39:00 on dist-test-slave-xt4k"
server_key: "420608baa3ccb7e677dac70e1056fb9d"
server_key_iv: "d8701e6ed1eb38b1014517d224f21f0f"
server_key_version: "encryptionkey@0"
I20260806 04:39:00.878127 13352 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:39:00.880560 13374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:00.881848 13352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:39:00.881973 13352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "88b82878b1c649da8ad6ff229c2bad47"
format_stamp: "Formatted at 2026-08-06 04:39:00 on dist-test-slave-xt4k"
server_key: "420608baa3ccb7e677dac70e1056fb9d"
server_key_iv: "d8701e6ed1eb38b1014517d224f21f0f"
server_key_version: "encryptionkey@0"
I20260806 04:39:00.882078 13352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:00.886643 13352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:00.889643 13352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:00.889828 13352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:00.896811 13352 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:35415
I20260806 04:39:00.896946 13456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:35415 every 8 connection(s)
I20260806 04:39:00.897790 13352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:39:00.900722 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:00.903825 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13352
I20260806 04:39:00.903946 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:39:00.904211 12351 external_mini_cluster.cc:1428] Setting key 682c229089e69dcc5df0ed243a7cd1b7
I20260806 04:39:00.906584 13457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47: Bootstrap starting.
I20260806 04:39:00.909090 13457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:00.909205 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:00.905586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43603 (local address 127.12.15.254:35415)
0806 04:39:00.905996 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:00.906011 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:00.906047 (+ 36us) server_negotiation.cc:407] Connection header received
0806 04:39:00.906655 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:00.906672 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:00.906966 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:00.907236 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:00.908029 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:00.908033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:00.908045 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:39:00.908055 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:00.908107 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:39:00.908126 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:00.908229 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:00.908316 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 04:39:00.908526 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
I20260806 04:39:00.909987 13457 log.cc:824] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:00.912092 13457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47: No bootstrap required, opened a new log
I20260806 04:39:00.912093 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:39:00.915648 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:00.914089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45781 (local address 127.12.15.254:35415)
0806 04:39:00.914189 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:00.914192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:00.914407 (+ 215us) server_negotiation.cc:407] Connection header received
0806 04:39:00.914521 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:00.914524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:00.914558 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:00.914622 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:00.915295 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:00.915298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:00.915301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:39:00.915313 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:00.915341 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:39:00.915351 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:00.915420 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:00.915495 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 04:39:00.915533 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 04:39:00.915891 13457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b82878b1c649da8ad6ff229c2bad47" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35415 } }
I20260806 04:39:00.916112 13457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:39:00.916162 13457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88b82878b1c649da8ad6ff229c2bad47, State: Initialized, Role: FOLLOWER
I20260806 04:39:00.916632 13457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b82878b1c649da8ad6ff229c2bad47" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35415 } }
I20260806 04:39:00.916800 13457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:39:00.916865 13457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:39:00.916962 13457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:00.917847 13457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b82878b1c649da8ad6ff229c2bad47" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35415 } }
I20260806 04:39:00.918176 13457 leader_election.cc:302] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88b82878b1c649da8ad6ff229c2bad47; no voters:
I20260806 04:39:00.918464 13457 leader_election.cc:288] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:39:00.918576 13470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:39:00.918783 13470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [term 1 LEADER]: Becoming Leader. State: Replica: 88b82878b1c649da8ad6ff229c2bad47, State: Running, Role: LEADER
I20260806 04:39:00.919131 13470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b82878b1c649da8ad6ff229c2bad47" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35415 } }
I20260806 04:39:00.919668 13457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:00.920789 13473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88b82878b1c649da8ad6ff229c2bad47. Latest consensus state: current_term: 1 leader_uuid: "88b82878b1c649da8ad6ff229c2bad47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b82878b1c649da8ad6ff229c2bad47" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35415 } } }
I20260806 04:39:00.920854 13471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88b82878b1c649da8ad6ff229c2bad47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b82878b1c649da8ad6ff229c2bad47" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 35415 } } }
I20260806 04:39:00.920998 13471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:00.921578 13483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:39:00.920995 13473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:00.924541 13483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:39:00.930001 13483 catalog_manager.cc:1347] Generated new cluster ID: e52e9ef5123a48ab85463bebf5419d55
I20260806 04:39:00.930084 13483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:39:00.947970 13483 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:39:00.948292 13483 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:39:00.953604 13483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88b82878b1c649da8ad6ff229c2bad47: Generated new TSK 0
I20260806 04:39:00.954241 13483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:39:00.963665 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13352
2026-08-06T04:39:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:39:01 dist-test-slave-xt4k krb5kdc[13503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:39:01 dist-test-slave-xt4k krb5kdc[13503](info): set up 2 sockets
Aug 06 04:39:01 dist-test-slave-xt4k krb5kdc[13503](info): commencing operation
krb5kdc: starting...
W20260806 04:39:02.106753 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:39:02 dist-test-slave-xt4k krb5kdc[13503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:39:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:39:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:39:02.227380 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:42385
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:42385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:39:02.340068 13517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:02.340358 13517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:02.340417 13517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:02.344414 13517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:02.344473 13517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:02.344491 13517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:02.344506 13517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:02.344520 13517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:02.349700 13517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:42385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:42385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:02.350692 13517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:02.351681 13517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:02.359864 13530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:02.359882 13524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:02.359916 13517 server_base.cc:1034] running on GCE node
W20260806 04:39:02.359876 13522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:02.360621 13517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:02.361680 13517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:02.362862 13517 hybrid_clock.cc:648] HybridClock initialized: now 1785991142362842 us; error 30 us; skew 500 ppm
Aug 06 04:39:02 dist-test-slave-xt4k krb5kdc[13503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991142, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:02.365681 13517 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:39:02.366586 13517 webserver.cc:460] Webserver started at http://127.12.15.254:40305/ using document root <none> and password file <none>
I20260806 04:39:02.367125 13517 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:02.367185 13517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:02.367381 13517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:02.369482 13517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "00e0cab1ec324557a74a184f7e299f9e"
format_stamp: "Formatted at 2026-08-06 04:39:02 on dist-test-slave-xt4k"
server_key: "9f46e3f9cc29a27c502a4291c353a9fb"
server_key_iv: "3bc67bbcb50092fc8bf03c32211fdd2a"
server_key_version: "encryptionkey@0"
I20260806 04:39:02.370025 13517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "00e0cab1ec324557a74a184f7e299f9e"
format_stamp: "Formatted at 2026-08-06 04:39:02 on dist-test-slave-xt4k"
server_key: "9f46e3f9cc29a27c502a4291c353a9fb"
server_key_iv: "3bc67bbcb50092fc8bf03c32211fdd2a"
server_key_version: "encryptionkey@0"
I20260806 04:39:02.373720 13517 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 04:39:02.376150 13540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:02.377413 13517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:39:02.377556 13517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "00e0cab1ec324557a74a184f7e299f9e"
format_stamp: "Formatted at 2026-08-06 04:39:02 on dist-test-slave-xt4k"
server_key: "9f46e3f9cc29a27c502a4291c353a9fb"
server_key_iv: "3bc67bbcb50092fc8bf03c32211fdd2a"
server_key_version: "encryptionkey@0"
I20260806 04:39:02.377656 13517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:02.400938 13517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:02.404027 13517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:02.404208 13517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:02.411120 13517 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:42385
I20260806 04:39:02.411190 13620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:42385 every 8 connection(s)
I20260806 04:39:02.412075 13517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:39:02.414469 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13517
I20260806 04:39:02.414616 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:39:02.414891 12351 external_mini_cluster.cc:1428] Setting key b56cc9d3e60388567a0068bbe97983d1
I20260806 04:39:02.414973 13621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:02.422224 13621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e: Bootstrap starting.
Aug 06 04:39:02 dist-test-slave-xt4k krb5kdc[13503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:39:02.424685 13621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:02.425585 13621 log.cc:824] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:02.427836 13621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e: No bootstrap required, opened a new log
I20260806 04:39:02.429813 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:02.416277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45263 (local address 127.12.15.254:42385)
0806 04:39:02.416754 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:02.416773 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:02.416815 (+ 42us) server_negotiation.cc:407] Connection header received
0806 04:39:02.417720 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:02.417744 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:02.418177 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:02.418553 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:02.419202 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:02.420677 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:02.421536 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:02.422202 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:02.422368 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:02.424892 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:02.424899 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:02.424917 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:02.424951 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:02.426667 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:02.427185 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:02.427191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:02.427197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:02.427333 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:02.427639 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:02.427642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:02.427644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:02.428141 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:02.428362 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:02.428596 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 04:39:02.428883 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":135,"threads_started":1}
I20260806 04:39:02.432113 13621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e0cab1ec324557a74a184f7e299f9e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 42385 } }
I20260806 04:39:02.432384 13621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:39:02.432437 13621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00e0cab1ec324557a74a184f7e299f9e, State: Initialized, Role: FOLLOWER
I20260806 04:39:02.432788 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:39:02.432893 13621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e0cab1ec324557a74a184f7e299f9e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 42385 } }
I20260806 04:39:02.433053 13621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:39:02.433108 13621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:39:02.433197 13621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:02.434322 13621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e0cab1ec324557a74a184f7e299f9e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 42385 } }
I20260806 04:39:02.434689 13621 leader_election.cc:302] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00e0cab1ec324557a74a184f7e299f9e; no voters:
I20260806 04:39:02.435029 13621 leader_election.cc:288] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:39:02.435132 13635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:39:02.435353 13635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [term 1 LEADER]: Becoming Leader. State: Replica: 00e0cab1ec324557a74a184f7e299f9e, State: Running, Role: LEADER
I20260806 04:39:02.435810 13635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e0cab1ec324557a74a184f7e299f9e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 42385 } }
I20260806 04:39:02.436380 13621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:02.437065 13635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00e0cab1ec324557a74a184f7e299f9e. Latest consensus state: current_term: 1 leader_uuid: "00e0cab1ec324557a74a184f7e299f9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e0cab1ec324557a74a184f7e299f9e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 42385 } } }
I20260806 04:39:02.437780 13635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:02.437672 13637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00e0cab1ec324557a74a184f7e299f9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e0cab1ec324557a74a184f7e299f9e" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 42385 } } }
I20260806 04:39:02.438067 13637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:02.438362 13650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:39:02.441672 13650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:39:02.443401 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:02.434985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57651 (local address 127.12.15.254:42385)
0806 04:39:02.435145 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:02.435149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:02.435295 (+ 146us) server_negotiation.cc:407] Connection header received
0806 04:39:02.435460 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:02.435464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:02.435633 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:02.435700 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:02.436629 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:02.437414 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:02.438524 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:02.438853 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:02.438949 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:02.440569 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:02.440573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:02.440575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:02.440599 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:02.441901 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:02.442291 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:02.442294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:02.442296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:02.442375 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:02.442630 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:02.442633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:02.442635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:02.442807 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:02.442921 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:02.443150 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 04:39:02.443258 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:39:02.447253 13650 catalog_manager.cc:1347] Generated new cluster ID: 7e8d031ab6ed4de2bdb53860f93d9c44
I20260806 04:39:02.447304 13650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:39:02.454721 13650 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:39:02.455533 13650 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:39:02.463249 13650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00e0cab1ec324557a74a184f7e299f9e: Generated new TSK 0
I20260806 04:39:02.463829 13650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:39:02.469689 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13517
2026-08-06T04:39:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:39:02 dist-test-slave-xt4k krb5kdc[13666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:39:02 dist-test-slave-xt4k krb5kdc[13666](info): set up 2 sockets
Aug 06 04:39:02 dist-test-slave-xt4k krb5kdc[13666](info): commencing operation
krb5kdc: starting...
W20260806 04:39:03.611374 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:39:03 dist-test-slave-xt4k krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:39:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:39:03.731787 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34755
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:46455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:39:03.842329 13685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:03.842625 13685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:03.842687 13685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:03.846696 13685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:03.846764 13685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:03.846791 13685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:03.846813 13685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:03.846837 13685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:03.852185 13685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:46455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:34755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:03.853331 13685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:03.854341 13685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:03.863001 13691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:03.863144 13685 server_base.cc:1034] running on GCE node
W20260806 04:39:03.862995 13692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:03.863001 13694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:03.863927 13685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:03.865072 13685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:03.866277 13685 hybrid_clock.cc:648] HybridClock initialized: now 1785991143866250 us; error 38 us; skew 500 ppm
Aug 06 04:39:03 dist-test-slave-xt4k krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:03.869161 13685 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:39:03.870036 13685 webserver.cc:460] Webserver started at http://127.12.15.254:43383/ using document root <none> and password file <none>
I20260806 04:39:03.870584 13685 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:03.870647 13685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:03.870895 13685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:03.873066 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "1e3189b387dc46a1b1b08f21a45bd8d6"
format_stamp: "Formatted at 2026-08-06 04:39:03 on dist-test-slave-xt4k"
server_key: "72f30a6688622ce18e1bbd9ce21aade5"
server_key_iv: "0ee0539b5101cf149ce9a6b3a08ee58c"
server_key_version: "encryptionkey@0"
I20260806 04:39:03.873625 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "1e3189b387dc46a1b1b08f21a45bd8d6"
format_stamp: "Formatted at 2026-08-06 04:39:03 on dist-test-slave-xt4k"
server_key: "72f30a6688622ce18e1bbd9ce21aade5"
server_key_iv: "0ee0539b5101cf149ce9a6b3a08ee58c"
server_key_version: "encryptionkey@0"
I20260806 04:39:03.877409 13685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 04:39:03.879896 13701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:03.881217 13685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:39:03.881335 13685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "1e3189b387dc46a1b1b08f21a45bd8d6"
format_stamp: "Formatted at 2026-08-06 04:39:03 on dist-test-slave-xt4k"
server_key: "72f30a6688622ce18e1bbd9ce21aade5"
server_key_iv: "0ee0539b5101cf149ce9a6b3a08ee58c"
server_key_version: "encryptionkey@0"
I20260806 04:39:03.881440 13685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:03.896231 13685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:03.899529 13685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:03.899722 13685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:03.906749 13685 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:34755
I20260806 04:39:03.906785 13783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:34755 every 8 connection(s)
I20260806 04:39:03.907717 13685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:39:03.908207 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13685
I20260806 04:39:03.908386 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:39:03.908650 12351 external_mini_cluster.cc:1428] Setting key 58d9204ca24806cba43197b6c83087cf
I20260806 04:39:03.910926 13784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:03.916687 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6: Bootstrap starting.
I20260806 04:39:03.917436 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:03.910161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49721 (local address 127.12.15.254:34755)
0806 04:39:03.910613 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:03.910627 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:03.910670 (+ 43us) server_negotiation.cc:407] Connection header received
0806 04:39:03.911340 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:03.911364 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:03.911731 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:03.912036 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:03.912483 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:03.913480 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:03.914329 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:03.914813 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:03.914963 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:03.915806 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:03.915811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:03.915824 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:39:03.916205 (+ 381us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:03.916260 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:39:03.916283 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:03.916387 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:03.916532 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 04:39:03.916726 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":139,"threads_started":1}
I20260806 04:39:03.919188 13784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:03.919955 13784 log.cc:824] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:03.920082 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:39:03.922062 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6: No bootstrap required, opened a new log
I20260806 04:39:03.925458 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:03.922090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49219 (local address 127.12.15.254:34755)
0806 04:39:03.922255 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:03.922259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:03.922300 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:39:03.922434 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:03.922437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:03.922486 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:03.922549 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:03.922793 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:03.923270 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:03.924012 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:03.924227 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:03.924312 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:03.925086 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:03.925090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:03.925092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:39:03.925104 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:03.925129 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:39:03.925139 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:03.925197 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:03.925282 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 04:39:03.925333 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 04:39:03.925890 13784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34755 } }
I20260806 04:39:03.926117 13784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:39:03.926183 13784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3189b387dc46a1b1b08f21a45bd8d6, State: Initialized, Role: FOLLOWER
I20260806 04:39:03.926625 13784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34755 } }
I20260806 04:39:03.926759 13784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:39:03.926824 13784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:39:03.926923 13784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:03.927765 13784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34755 } }
I20260806 04:39:03.928085 13784 leader_election.cc:302] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3189b387dc46a1b1b08f21a45bd8d6; no voters:
I20260806 04:39:03.928386 13784 leader_election.cc:288] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:39:03.928475 13796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:39:03.928690 13796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e3189b387dc46a1b1b08f21a45bd8d6, State: Running, Role: LEADER
I20260806 04:39:03.929064 13796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34755 } }
I20260806 04:39:03.929744 13784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:03.930644 13797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34755 } } }
I20260806 04:39:03.930776 13797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:03.931159 13809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:39:03.931121 13796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e3189b387dc46a1b1b08f21a45bd8d6. Latest consensus state: current_term: 1 leader_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3189b387dc46a1b1b08f21a45bd8d6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 34755 } } }
I20260806 04:39:03.931258 13796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:03.934369 13809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:39:03.940135 13809 catalog_manager.cc:1347] Generated new cluster ID: 63c98d87112e4666931383da0f5eaf71
I20260806 04:39:03.940198 13809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:39:03.954530 13809 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:39:03.955608 13809 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:39:03.968108 13809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e3189b387dc46a1b1b08f21a45bd8d6: Generated new TSK 0
I20260806 04:39:03.968806 13809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:39:03.977921 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13685
2026-08-06T04:39:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:39:04 dist-test-slave-xt4k krb5kdc[13825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:39:04 dist-test-slave-xt4k krb5kdc[13825](info): set up 2 sockets
Aug 06 04:39:04 dist-test-slave-xt4k krb5kdc[13825](info): commencing operation
krb5kdc: starting...
W20260806 04:39:05.124301 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:39:05 dist-test-slave-xt4k krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:39:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:39:05.254997 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33665
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:33665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:39:05.369876 13846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:05.370155 13846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:05.370196 13846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:05.374305 13846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:05.374373 13846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:05.374392 13846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:05.374411 13846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:05.374425 13846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:05.379799 13846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:33665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:33665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:05.380993 13846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:05.382156 13846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:39:05.390571 13846 server_base.cc:1034] running on GCE node
W20260806 04:39:05.390508 13855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:05.390483 13853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:05.390502 13858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:05.391361 13846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:05.392509 13846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:05.393695 13846 hybrid_clock.cc:648] HybridClock initialized: now 1785991145393664 us; error 42 us; skew 500 ppm
Aug 06 04:39:05 dist-test-slave-xt4k krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991145, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:05.396420 13846 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:39:05.397353 13846 webserver.cc:460] Webserver started at http://127.12.15.254:38627/ using document root <none> and password file <none>
I20260806 04:39:05.397897 13846 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:05.397961 13846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:05.398169 13846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:05.400177 13846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "9e91cef598e94ec2935218a1b22696b6"
format_stamp: "Formatted at 2026-08-06 04:39:05 on dist-test-slave-xt4k"
server_key: "b6cbc1631a6f82b43f421752035b3cfa"
server_key_iv: "1f3b6708ef853e800dd11251160c9949"
server_key_version: "encryptionkey@0"
I20260806 04:39:05.400758 13846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "9e91cef598e94ec2935218a1b22696b6"
format_stamp: "Formatted at 2026-08-06 04:39:05 on dist-test-slave-xt4k"
server_key: "b6cbc1631a6f82b43f421752035b3cfa"
server_key_iv: "1f3b6708ef853e800dd11251160c9949"
server_key_version: "encryptionkey@0"
I20260806 04:39:05.404495 13846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:39:05.406812 13866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:05.408015 13846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:39:05.408151 13846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "9e91cef598e94ec2935218a1b22696b6"
format_stamp: "Formatted at 2026-08-06 04:39:05 on dist-test-slave-xt4k"
server_key: "b6cbc1631a6f82b43f421752035b3cfa"
server_key_iv: "1f3b6708ef853e800dd11251160c9949"
server_key_version: "encryptionkey@0"
I20260806 04:39:05.408244 13846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:05.412742 13846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:05.415725 13846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:05.415894 13846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:05.422770 13846 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:33665
I20260806 04:39:05.422791 13937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:33665 every 8 connection(s)
I20260806 04:39:05.423694 13846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:39:05.426610 13938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:05.431921 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 13846
I20260806 04:39:05.432076 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:39:05.432268 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6: Bootstrap starting.
I20260806 04:39:05.432367 12351 external_mini_cluster.cc:1428] Setting key 9ce1eb493045a89e15683d78297116d0
I20260806 04:39:05.434852 13938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:05.435685 13938 log.cc:824] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:05.437638 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:05.433712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42729 (local address 127.12.15.254:33665)
0806 04:39:05.434151 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:05.434166 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:05.434201 (+ 35us) server_negotiation.cc:407] Connection header received
0806 04:39:05.434820 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:05.434836 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:05.435165 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:05.435444 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:05.436322 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:05.436329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:05.436369 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:39:05.436383 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:05.436451 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:39:05.436481 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:05.436598 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:05.436740 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 04:39:05.436899 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
I20260806 04:39:05.438156 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6: No bootstrap required, opened a new log
I20260806 04:39:05.440611 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:39:05.442179 13938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e91cef598e94ec2935218a1b22696b6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33665 } }
I20260806 04:39:05.442425 13938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 04:39:05.442513 13938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e91cef598e94ec2935218a1b22696b6, State: Initialized, Role: FOLLOWER
I20260806 04:39:05.443039 13938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e91cef598e94ec2935218a1b22696b6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33665 } }
I20260806 04:39:05.443218 13938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 04:39:05.443290 13938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 04:39:05.443377 13938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:05.444109 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:05.442603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43465 (local address 127.12.15.254:33665)
0806 04:39:05.442717 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:05.442721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:05.442863 (+ 142us) server_negotiation.cc:407] Connection header received
0806 04:39:05.442964 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:05.442967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:05.443004 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:05.443074 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:05.443759 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:05.443763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:05.443765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 04:39:05.443776 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:05.443807 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 04:39:05.443818 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:05.443904 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:05.443959 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 04:39:05.443996 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 04:39:05.444553 13938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e91cef598e94ec2935218a1b22696b6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33665 } }
I20260806 04:39:05.444906 13938 leader_election.cc:302] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e91cef598e94ec2935218a1b22696b6; no voters:
I20260806 04:39:05.445207 13938 leader_election.cc:288] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 04:39:05.445297 13955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:39:05.445565 13955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [term 1 LEADER]: Becoming Leader. State: Replica: 9e91cef598e94ec2935218a1b22696b6, State: Running, Role: LEADER
I20260806 04:39:05.445981 13955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e91cef598e94ec2935218a1b22696b6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33665 } }
I20260806 04:39:05.446429 13938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:05.447609 13958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e91cef598e94ec2935218a1b22696b6. Latest consensus state: current_term: 1 leader_uuid: "9e91cef598e94ec2935218a1b22696b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e91cef598e94ec2935218a1b22696b6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33665 } } }
I20260806 04:39:05.447739 13958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:05.448071 13956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e91cef598e94ec2935218a1b22696b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e91cef598e94ec2935218a1b22696b6" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 33665 } } }
I20260806 04:39:05.448129 13967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:39:05.449105 13956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:05.450973 13967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:39:05.455879 13967 catalog_manager.cc:1347] Generated new cluster ID: 5c80d3d8baa34f989437f9a7b87e4a8c
I20260806 04:39:05.455948 13967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:39:05.468060 13967 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:39:05.468405 13967 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:39:05.472886 13967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e91cef598e94ec2935218a1b22696b6: Generated new TSK 0
I20260806 04:39:05.473532 13967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 04:39:05.481227 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 13846
2026-08-06T04:39:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1507 ms)
[----------] 18 tests from AuthTokenIssuingTest (27331 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:39:05 dist-test-slave-xt4k krb5kdc[13986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:39:05 dist-test-slave-xt4k krb5kdc[13986](info): set up 2 sockets
Aug 06 04:39:05 dist-test-slave-xt4k krb5kdc[13986](info): commencing operation
krb5kdc: starting...
W20260806 04:39:06.623831 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:39:06 dist-test-slave-xt4k krb5kdc[13986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:39:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:39:06.752260 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:46295
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:46295,127.12.15.253:42751,127.12.15.252:41741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:39:06.868108 14004 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 04:39:06.868409 14004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:06.868446 14004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:06.868484 14004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:06.872328 14004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:06.872400 14004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:06.872417 14004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:06.872432 14004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:06.872445 14004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:06.877555 14004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:46295,127.12.15.253:42751,127.12.15.252:41741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:46295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:06.878525 14004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:06.879472 14004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:06.887754 14013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:06.887890 14004 server_base.cc:1034] running on GCE node
W20260806 04:39:06.887724 14015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:06.887735 14012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:06.888689 14004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:06.889773 14004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:06.890950 14004 hybrid_clock.cc:648] HybridClock initialized: now 1785991146890931 us; error 33 us; skew 500 ppm
Aug 06 04:39:06 dist-test-slave-xt4k krb5kdc[13986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:06.896853 14004 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:39:06.897779 14004 webserver.cc:460] Webserver started at http://127.12.15.254:43359/ using document root <none> and password file <none>
I20260806 04:39:06.898327 14004 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:06.898393 14004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:06.898639 14004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:06.900753 14004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "b8b4e630043949719ec55667d328d310"
format_stamp: "Formatted at 2026-08-06 04:39:06 on dist-test-slave-xt4k"
server_key: "1c9fa98af8be54e6fcf362ca186de770"
server_key_iv: "84bd8c8fcae29da1f7e6460d8e13fea3"
server_key_version: "encryptionkey@0"
I20260806 04:39:06.901299 14004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "b8b4e630043949719ec55667d328d310"
format_stamp: "Formatted at 2026-08-06 04:39:06 on dist-test-slave-xt4k"
server_key: "1c9fa98af8be54e6fcf362ca186de770"
server_key_iv: "84bd8c8fcae29da1f7e6460d8e13fea3"
server_key_version: "encryptionkey@0"
I20260806 04:39:06.904984 14004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:39:06.907258 14022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:06.908497 14004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:39:06.908628 14004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "b8b4e630043949719ec55667d328d310"
format_stamp: "Formatted at 2026-08-06 04:39:06 on dist-test-slave-xt4k"
server_key: "1c9fa98af8be54e6fcf362ca186de770"
server_key_iv: "84bd8c8fcae29da1f7e6460d8e13fea3"
server_key_version: "encryptionkey@0"
I20260806 04:39:06.908722 14004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:06.918969 14004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:06.921627 14004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:06.921782 14004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:06.928237 14004 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:46295
I20260806 04:39:06.928539 14107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:46295 every 8 connection(s)
I20260806 04:39:06.929198 14004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:39:06.931936 14108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:06.935619 14108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:06.939266 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14004
I20260806 04:39:06.939388 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:39:06.939690 12351 external_mini_cluster.cc:1428] Setting key 36b583a0d2947eccd6d948e03247cd5a
Aug 06 04:39:06 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.253@KRBTEST.COM; defaulting to no policy
I20260806 04:39:06.950872 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:06.937900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.12.15.254:46295)
0806 04:39:06.938408 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:06.938457 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:06.938597 (+ 140us) server_negotiation.cc:407] Connection header received
0806 04:39:06.939635 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:06.939642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:06.939904 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:06.940100 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:06.940888 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:06.942080 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:06.943435 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:06.943791 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:06.943915 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:06.946443 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:06.946449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:06.946459 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:06.946480 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:06.947944 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:06.948404 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:06.948411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:06.948412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:06.948531 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:06.948828 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:06.948832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:06.948834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:06.949195 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:06.949352 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:06.949685 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 04:39:06.949874 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":389,"spinlock_wait_cycles":45824,"thread_start_us":140,"threads_started":1}
I20260806 04:39:06.950870 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:06.937907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:46295 (local address 127.0.0.1:33628)
0806 04:39:06.938428 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:06.938480 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:06.939298 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:06.940110 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:06.940115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:06.940478 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:06.940741 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:06.940754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:06.942209 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:06.942212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:06.943299 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:06.943310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:06.943947 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:06.943951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:06.944107 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:06.944668 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:06.944688 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:06.946301 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:06.948043 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:06.948053 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:06.948063 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:06.948286 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:06.948622 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:06.948625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:06.948627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:06.948735 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:06.949304 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:06.949310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:06.949453 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 04:39:06.949592 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 04:39:06.949944 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":88,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.253@KRBTEST.COM" created.
I20260806 04:39:06.953486 14108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:06.954156 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:06.953920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59432)
0806 04:39:06.954030 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:06.954097 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 04:39:06.954329 14029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
I20260806 04:39:06.955127 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:06.954937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59438)
0806 04:39:06.955030 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:06.955074 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 04:39:06.955243 14108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:42751: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab.
Entry for principal kudu/127.12.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab.
I20260806 04:39:06.965296 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:42751
--webserver_interface=127.12.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab
--principal=kudu/127.12.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:46295,127.12.15.253:42751,127.12.15.252:41741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:39:07.004465 14108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } attempt: 1
I20260806 04:39:07.005271 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.004989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59442)
0806 04:39:07.005129 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.005190 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 04:39:07.005926 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.005719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59448)
0806 04:39:07.005823 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.005868 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 04:39:07.006057 14108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:42751: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
I20260806 04:39:07.074205 14108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } attempt: 2
I20260806 04:39:07.075070 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.074763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59456)
0806 04:39:07.074918 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.074984 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 04:39:07.075788 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.075564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59460)
0806 04:39:07.075665 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.075716 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 04:39:07.075965 14108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:42751: Network error: Client connection negotiation failed: client connection to 127.12.15.253:42751: connect: Connection refused (error 111)
W20260806 04:39:07.080924 14117 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 04:39:07.081211 14117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:07.081255 14117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:07.081321 14117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:07.085781 14117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:07.085847 14117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:07.085873 14117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:07.085898 14117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:07.085928 14117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:07.091529 14117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:46295,127.12.15.253:42751,127.12.15.252:41741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab
--principal=kudu/127.12.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:42751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:07.092733 14117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:07.093801 14117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:07.102550 14123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:07.102671 14117 server_base.cc:1034] running on GCE node
W20260806 04:39:07.102551 14126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:07.102551 14124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:07.103485 14117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:07.104709 14117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:07.105892 14117 hybrid_clock.cc:648] HybridClock initialized: now 1785991147105872 us; error 34 us; skew 500 ppm
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:07.109041 14117 init.cc:377] Logged in from keytab as kudu/127.12.15.253@KRBTEST.COM (short username kudu)
I20260806 04:39:07.110095 14117 webserver.cc:460] Webserver started at http://127.12.15.253:44649/ using document root <none> and password file <none>
I20260806 04:39:07.110688 14117 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:07.110754 14117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:07.110958 14117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:07.113180 14117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data/instance:
uuid: "dc4a5f25e7af4e909f505de9f75125ae"
format_stamp: "Formatted at 2026-08-06 04:39:07 on dist-test-slave-xt4k"
server_key: "0d844dbca8f55001a449b633ce5b34bb"
server_key_iv: "0a093f61ce0c85725eac0e94c2876995"
server_key_version: "encryptionkey@0"
I20260806 04:39:07.113730 14117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal/instance:
uuid: "dc4a5f25e7af4e909f505de9f75125ae"
format_stamp: "Formatted at 2026-08-06 04:39:07 on dist-test-slave-xt4k"
server_key: "0d844dbca8f55001a449b633ce5b34bb"
server_key_iv: "0a093f61ce0c85725eac0e94c2876995"
server_key_version: "encryptionkey@0"
I20260806 04:39:07.117544 14117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 04:39:07.120085 14135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:07.121320 14117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 04:39:07.121435 14117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal
uuid: "dc4a5f25e7af4e909f505de9f75125ae"
format_stamp: "Formatted at 2026-08-06 04:39:07 on dist-test-slave-xt4k"
server_key: "0d844dbca8f55001a449b633ce5b34bb"
server_key_iv: "0a093f61ce0c85725eac0e94c2876995"
server_key_version: "encryptionkey@0"
I20260806 04:39:07.121541 14117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:07.146281 14117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:07.149475 14117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:07.149669 14117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:07.156804 14117 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.253:42751
I20260806 04:39:07.157135 14211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.253:42751 every 8 connection(s)
I20260806 04:39:07.157791 14117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
I20260806 04:39:07.160658 14212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:07.161895 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14117
I20260806 04:39:07.162009 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-1/wal/instance
I20260806 04:39:07.162272 12351 external_mini_cluster.cc:1428] Setting key 27ae679682df7a2b8e639c19e4711e91
I20260806 04:39:07.164533 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.167122 14108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } attempt: 3
WARNING: no policy specified for kudu/127.12.15.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.252@KRBTEST.COM" created.
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:39:07.179570 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.167539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59468)
0806 04:39:07.167698 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.167712 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.167789 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.168850 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.168853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.169056 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.169188 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.169196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.170461 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.170465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.171284 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.171295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.172012 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.172033 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.172123 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.172650 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.172666 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.175031 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.177186 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.177190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.177192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.177533 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.177899 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.177902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.177904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.177996 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.178988 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.178992 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.179117 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.179234 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.179391 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 04:39:07.180050 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.166414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.12.15.254:46295)
0806 04:39:07.166582 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.166587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.167372 (+ 785us) server_negotiation.cc:407] Connection header received
0806 04:39:07.168512 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.168515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.168570 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.168662 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.169680 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.170460 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.171721 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.172074 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.172173 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.175426 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.175430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.175433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.175459 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.177676 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.178180 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.178184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.178185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.178271 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.178688 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.178692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.178694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.178932 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.179104 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.179665 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.179908 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 04:39:07.180840 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.167882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.12.15.253:42751)
0806 04:39:07.168259 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.168272 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.168313 (+ 41us) server_negotiation.cc:407] Connection header received
0806 04:39:07.168482 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.168487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.168698 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.168866 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.169370 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.170350 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.171418 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.171884 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.172058 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.175174 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.175180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.175191 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.175214 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.177063 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.177642 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.177648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.177654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.177778 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.178126 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.178129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.178131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.178860 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.179011 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.179385 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.180478 (+ 1093us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":276,"thread_start_us":168,"threads_started":1}
I20260806 04:39:07.180843 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.166752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:46295 (local address 127.0.0.1:33636)
0806 04:39:07.167224 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.167276 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.168271 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.168716 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.168725 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.169154 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.169533 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.169555 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.170577 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.170581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.171584 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.171594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.172237 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.172241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.172388 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.172850 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.172871 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.175274 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.177776 (+ 2502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.177785 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.177792 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.178082 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.178373 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.178376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.178377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.178460 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.179096 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.179102 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.179252 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.179405 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.179752 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":41,"thread_start_us":120,"threads_started":1}
I20260806 04:39:07.182178 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.183188 14108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.183852 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.183593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59876)
0806 04:39:07.183717 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.183776 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 04:39:07.184635 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.184382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59886)
0806 04:39:07.184522 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.184576 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 04:39:07.184793 14108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:41741: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
I20260806 04:39:07.191048 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.182688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59478)
0806 04:39:07.182824 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.182837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.182922 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.183179 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.183182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.183356 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.183463 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.183470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.184329 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.184356 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.185192 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.185203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.185648 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.185651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.185724 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.186163 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.186175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.188054 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.189569 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.189572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.189574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.189758 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.190094 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.190097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.190098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.190178 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.190597 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.190599 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.190688 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.190780 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.190897 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 04:39:07.191092 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.182697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.12.15.253:42751)
0806 04:39:07.182816 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.182819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.182988 (+ 169us) server_negotiation.cc:407] Connection header received
0806 04:39:07.183012 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.183014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.183050 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.183129 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.183608 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.184227 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.185309 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.185537 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.185626 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.188188 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.188191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.188193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.188215 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.189446 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.189894 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.189915 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.189916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.190004 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.190279 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.190282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.190283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.190477 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.190577 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.190833 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.190963 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":55}
I20260806 04:39:07.191963 14212 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.192729 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.192484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59902)
0806 04:39:07.192604 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.192678 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 04:39:07.192989 14140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
I20260806 04:39:07.194415 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.194200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59910)
0806 04:39:07.194301 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.194358 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab.
Entry for principal kudu/127.12.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab.
W20260806 04:39:07.194584 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:41741: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
I20260806 04:39:07.195326 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:41741
--webserver_interface=127.12.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab
--principal=kudu/127.12.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:46295,127.12.15.253:42751,127.12.15.252:41741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:39:07.215956 14108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } attempt: 1
I20260806 04:39:07.216861 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.216517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59924)
0806 04:39:07.216680 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.216762 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 04:39:07.217574 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.217357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59940)
0806 04:39:07.217453 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.217503 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 04:39:07.217728 14108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:41741: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
I20260806 04:39:07.243887 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } attempt: 1
I20260806 04:39:07.244868 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.244523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59950)
0806 04:39:07.244691 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.244769 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 04:39:07.245550 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.245337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59954)
0806 04:39:07.245431 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.245484 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 04:39:07.245668 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:41741: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
I20260806 04:39:07.292896 14108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } attempt: 2
I20260806 04:39:07.293816 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.293466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59968)
0806 04:39:07.293641 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.293717 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 04:39:07.294615 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.294383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59984)
0806 04:39:07.294496 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.294550 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 04:39:07.294812 14108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:41741: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
W20260806 04:39:07.311764 14223 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 04:39:07.312036 14223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:07.312058 14223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:07.312107 14223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 04:39:07.313814 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } attempt: 2
I20260806 04:39:07.314618 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.314314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59990)
0806 04:39:07.314461 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.314528 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 04:39:07.315294 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.315094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:59994)
0806 04:39:07.315183 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.315233 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 04:39:07.315404 14212 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:41741: Network error: Client connection negotiation failed: client connection to 127.12.15.252:41741: connect: Connection refused (error 111)
W20260806 04:39:07.316531 14223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:07.316578 14223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:07.316597 14223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:07.316617 14223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:07.316632 14223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:07.321980 14223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:46295,127.12.15.253:42751,127.12.15.252:41741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab
--principal=kudu/127.12.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:41741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:07.323033 14223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:07.324008 14223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:07.332326 14235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:07.332396 14223 server_base.cc:1034] running on GCE node
W20260806 04:39:07.332329 14229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:07.332326 14231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:07.333168 14223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:07.334517 14223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:07.335704 14223 hybrid_clock.cc:648] HybridClock initialized: now 1785991147335686 us; error 34 us; skew 500 ppm
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:07.338476 14223 init.cc:377] Logged in from keytab as kudu/127.12.15.252@KRBTEST.COM (short username kudu)
I20260806 04:39:07.339361 14223 webserver.cc:460] Webserver started at http://127.12.15.252:37123/ using document root <none> and password file <none>
I20260806 04:39:07.339905 14223 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:07.339970 14223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:07.340164 14223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:07.342267 14223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data/instance:
uuid: "805b874c61624f24baca92429580f852"
format_stamp: "Formatted at 2026-08-06 04:39:07 on dist-test-slave-xt4k"
server_key: "72674b4c22e4adf76311e503f7303097"
server_key_iv: "f165b5e357fdf8d161c128cc67d7fc0f"
server_key_version: "encryptionkey@0"
I20260806 04:39:07.342823 14223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal/instance:
uuid: "805b874c61624f24baca92429580f852"
format_stamp: "Formatted at 2026-08-06 04:39:07 on dist-test-slave-xt4k"
server_key: "72674b4c22e4adf76311e503f7303097"
server_key_iv: "f165b5e357fdf8d161c128cc67d7fc0f"
server_key_version: "encryptionkey@0"
I20260806 04:39:07.346484 14223 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 04:39:07.348989 14243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:07.350296 14223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 04:39:07.350417 14223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal
uuid: "805b874c61624f24baca92429580f852"
format_stamp: "Formatted at 2026-08-06 04:39:07 on dist-test-slave-xt4k"
server_key: "72674b4c22e4adf76311e503f7303097"
server_key_iv: "f165b5e357fdf8d161c128cc67d7fc0f"
server_key_version: "encryptionkey@0"
I20260806 04:39:07.350548 14223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:07.373056 14223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:07.375967 14223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:07.376156 14223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:07.382927 14223 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.252:41741
I20260806 04:39:07.383198 14315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.252:41741 every 8 connection(s)
I20260806 04:39:07.383903 14223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
I20260806 04:39:07.386679 14316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:07.390429 14316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.392716 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14223
I20260806 04:39:07.392834 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785990859032933-12351-0/minicluster-data/master-2/wal/instance
I20260806 04:39:07.393155 12351 external_mini_cluster.cc:1428] Setting key 584d616608ce87dd493bcf29dd1a1abd
I20260806 04:39:07.394100 14108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } attempt: 3
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
I20260806 04:39:07.405563 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.392314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.12.15.254:46295)
0806 04:39:07.392503 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.392506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.393408 (+ 902us) server_negotiation.cc:407] Connection header received
0806 04:39:07.394513 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.394517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.394580 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.394630 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.396025 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.396567 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.397931 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.398166 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.398335 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.401942 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.401945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.401947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.401968 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.403268 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.403849 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.403853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.403855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.403927 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404248 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.404251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.404253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.404577 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.404675 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.405278 (+ 603us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.405396 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 04:39:07.405843 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.394511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:60006)
0806 04:39:07.394673 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.394686 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.394740 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.395479 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.395482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.395678 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.395774 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.395780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.397484 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.397487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.398216 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.398228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.398990 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.399002 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.399105 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.399512 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.399527 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.400889 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.404006 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.404011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.404255 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.404583 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.404588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.404678 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.405188 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.405190 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.405292 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.405384 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.405530 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"spinlock_wait_cycles":2816}
I20260806 04:39:07.406561 14212 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } attempt: 3
I20260806 04:39:07.406953 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.394562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.12.15.252:41741)
0806 04:39:07.394847 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.394854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.394866 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:39:07.394894 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.394897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.394927 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.395106 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.395931 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.397367 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.398349 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.398851 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.398954 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.401025 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.401031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.401053 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.401080 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.403880 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404406 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.404412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.404414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.404493 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404779 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.404782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.404784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.405071 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.405197 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.405707 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.406006 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":521,"server-negotiator.queue_time_us":228,"spinlock_wait_cycles":3200,"thread_start_us":111,"threads_started":1}
I20260806 04:39:07.406960 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.392735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:46295 (local address 127.0.0.1:33650)
0806 04:39:07.393258 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.393312 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.394289 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.394903 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.394909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.395344 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.395893 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.395914 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.396673 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.396677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.397817 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.397828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.398372 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.398376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.398566 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.399388 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.399411 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.401810 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.403370 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.403379 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.403391 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.403758 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.404011 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.404016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.404159 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.404693 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.404700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.405006 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.406035 (+ 1029us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.406200 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":232,"thread_start_us":146,"threads_started":1}
I20260806 04:39:07.408138 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.393822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.12.15.252:41741)
0806 04:39:07.394474 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.394485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.394527 (+ 42us) server_negotiation.cc:407] Connection header received
0806 04:39:07.394696 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.394717 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.394929 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.395106 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.395931 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.397367 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.398994 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.399242 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.399326 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.401892 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.401896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.401899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.401924 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.404304 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.404751 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.404755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.404757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.404832 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.405072 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.405075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.405077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.405284 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.405458 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.406172 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.406277 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":360,"server-negotiator.queue_time_us":559,"spinlock_wait_cycles":16256,"thread_start_us":144,"threads_started":1}
I20260806 04:39:07.408778 14316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.411564 14108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310: Bootstrap starting.
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
I20260806 04:39:07.414366 14108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:07.415315 14108 log.cc:824] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:07.416738 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.407012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:60018)
0806 04:39:07.407196 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.407210 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.407289 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.407670 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.407673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.407808 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.407895 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.407900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.409845 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.409849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.410606 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.410615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.411173 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.411177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.411261 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.411741 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.411755 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.413242 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.415101 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.415104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.415107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.415353 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.415619 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.415622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.415624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.415708 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.416152 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.416155 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.416252 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.416434 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.416570 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 04:39:07.417248 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.407062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.12.15.252:41741)
0806 04:39:07.407461 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.407465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.407476 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:39:07.407518 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.407520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.407560 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.407632 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.408000 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.409740 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.410735 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.411059 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.411146 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.413390 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.413394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.413397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.413423 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.415004 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.415445 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.415449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.415451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.415538 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.415811 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.415814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.415816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.416030 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.416909 (+ 879us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.416981 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.417102 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":340,"thread_start_us":110,"threads_started":1}
I20260806 04:39:07.418062 14108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310: No bootstrap required, opened a new log
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:39:07.420387 14212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: Bootstrap starting.
I20260806 04:39:07.420909 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.409401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.12.15.253:42751)
0806 04:39:07.409506 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.409510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.409570 (+ 60us) server_negotiation.cc:407] Connection header received
0806 04:39:07.409703 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.409706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.409746 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.409792 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.410315 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.410987 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.411995 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.412259 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.412367 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.414398 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.414402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.414404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.414430 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.415904 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.416392 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.416395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.416397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.416484 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.416806 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.416809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.416811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.416996 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.417063 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.420639 (+ 3576us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.420755 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 04:39:07.421339 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.409255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:42751 (local address 127.0.0.1:59480)
0806 04:39:07.409512 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.409524 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.409622 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.409918 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.409921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.410090 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.410191 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.410197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.411129 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.411132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.411845 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.411855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.412379 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.412383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.412478 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.412853 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.412867 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.414254 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.416017 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.416020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.416022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.416245 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.416617 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.416619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.416621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.416705 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.417246 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.417249 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.420416 (+ 3167us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.421042 (+ 626us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.421184 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":20}
I20260806 04:39:07.422031 14316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:07.422128 14108 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.422339 14108 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b4e630043949719ec55667d328d310, State: Initialized, Role: FOLLOWER
I20260806 04:39:07.422681 14108 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.423166 14212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:07.423933 14108 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:07.424047 14212 log.cc:824] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:07.424230 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.410054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.12.15.254:46295)
0806 04:39:07.415534 (+ 5480us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.415538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.415553 (+ 15us) server_negotiation.cc:407] Connection header received
0806 04:39:07.415589 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.415592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.415636 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.415720 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.416515 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.417182 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.417992 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.418209 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.418293 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.420383 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.420387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.420390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.420415 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.422046 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.422517 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.422521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.422522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.422610 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.422927 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.422930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.422932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.423159 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.423309 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.423967 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.424084 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5395}
I20260806 04:39:07.426708 14212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: No bootstrap required, opened a new log
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991147, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:39:07 dist-test-slave-xt4k krb5kdc[13986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
I20260806 04:39:07.431591 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.422542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:41741 (local address 127.0.0.1:60032)
0806 04:39:07.423009 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:07.423021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:07.423551 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:07.423834 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:07.423837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:07.423991 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:07.424076 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.424083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.424995 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.424998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.425740 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:07.425751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.426325 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.426328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:07.426403 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.426816 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:07.426829 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:07.428279 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:07.430115 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.430117 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.430119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.430342 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.430626 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:07.430628 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:07.430631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:07.430704 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:07.431098 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:07.431100 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:07.431186 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 04:39:07.431278 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 04:39:07.431403 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":121,"threads_started":1}
I20260806 04:39:07.431746 14212 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.431967 14212 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc4a5f25e7af4e909f505de9f75125ae, State: Initialized, Role: FOLLOWER
I20260806 04:39:07.432071 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.422888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.12.15.252:41741)
0806 04:39:07.423518 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.423522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.423535 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:39:07.423640 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.423643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.423683 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.423742 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.424181 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.424858 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.425847 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.426140 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.426226 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.428426 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.428429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.428432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.428453 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.430008 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.430443 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.430446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.430448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.430539 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.430785 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.430787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.430789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.430973 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.431064 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.431834 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.431941 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260806 04:39:07.432548 14212 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.430155 14004 master_runner.cc:186] Error getting master registration for 127.12.15.253:42751: OK, instance_id { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" instance_seqno: 1785991147154776 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 04:39:07.433916 14004 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 04:39:07.434005 14212 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:07.435293 14316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: Bootstrap starting.
I20260806 04:39:07.435611 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.425489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.12.15.253:42751)
0806 04:39:07.425584 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.425588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.425763 (+ 175us) server_negotiation.cc:407] Connection header received
0806 04:39:07.425922 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.425925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.425963 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.426016 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.427102 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.427785 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.428840 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.429172 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.429245 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.432188 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.432192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.432195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.432222 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.433729 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.434231 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.434235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.434237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.434323 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.434627 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.434631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.434632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.434838 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.434986 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.435359 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.435472 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 04:39:07.435985 14329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.437570 14329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.438156 14316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:07.438884 14316 log.cc:824] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: Log is configured to *not* fsync() on all Append() calls
W20260806 04:39:07.439390 14353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:39:07.439462 14353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:39:07.440158 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:39:07.440585 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.440693 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.441607 14061 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b8b4e630043949719ec55667d328d310"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33658
I20260806 04:39:07.441931 14061 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 04:39:07.442076 14061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:07.443476 14061 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.444559 14061 leader_election.cc:288] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [CANDIDATE]: Term 1 election: Requested vote from peers dc4a5f25e7af4e909f505de9f75125ae (127.12.15.253:42751), 805b874c61624f24baca92429580f852 (127.12.15.252:41741)
I20260806 04:39:07.441764 14117 master_runner.cc:186] Error getting master registration for 127.12.15.252:41741: OK, instance_id { permanent_uuid: "805b874c61624f24baca92429580f852" instance_seqno: 1785991147381034 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 04:39:07.444864 14117 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 04:39:07.445295 14170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8b4e630043949719ec55667d328d310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dc4a5f25e7af4e909f505de9f75125ae"
I20260806 04:39:07.445538 14170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:07.446897 14170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8b4e630043949719ec55667d328d310 in term 1.
W20260806 04:39:07.447199 14367 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:39:07.447319 14367 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:39:07.447594 14029 leader_election.cc:302] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8b4e630043949719ec55667d328d310, dc4a5f25e7af4e909f505de9f75125ae; no voters:
I20260806 04:39:07.447928 14329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:39:07.448168 14329 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [term 1 LEADER]: Becoming Leader. State: Replica: b8b4e630043949719ec55667d328d310, State: Running, Role: LEADER
I20260806 04:39:07.449779 14316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: No bootstrap required, opened a new log
I20260806 04:39:07.451128 14329 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.453074 14329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8b4e630043949719ec55667d328d310. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.453199 14329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:07.454246 14316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.454568 14316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805b874c61624f24baca92429580f852, State: Initialized, Role: FOLLOWER
I20260806 04:39:07.455015 14316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } }
I20260806 04:39:07.456535 14316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:07.456624 14282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8b4e630043949719ec55667d328d310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "805b874c61624f24baca92429580f852"
I20260806 04:39:07.456864 14282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:07.458796 14282 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8b4e630043949719ec55667d328d310 in term 1.
I20260806 04:39:07.459045 14384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:39:07.460084 14384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 04:39:07.468253 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.455483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.12.15.252:41741)
0806 04:39:07.455592 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.455596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.458535 (+ 2939us) server_negotiation.cc:407] Connection header received
0806 04:39:07.458644 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.458647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.458692 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.458777 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.459734 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.460594 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.461808 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.462160 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.462231 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.464194 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.464198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.464200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.464228 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.466091 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.466882 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.466886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.466888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.466976 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.467273 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.467276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.467278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.467484 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.467613 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.467973 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.468095 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 04:39:07.469446 14387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.469583 14387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.474951 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.454583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.12.15.253:42751)
0806 04:39:07.454715 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.454720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.456865 (+ 2145us) server_negotiation.cc:407] Connection header received
0806 04:39:07.457140 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.457144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.457198 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.457253 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.458693 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.459387 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.469315 (+ 9928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.469636 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.469737 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.471288 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.471293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.471296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.471323 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.472935 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.473395 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.473399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.473401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.473482 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.473771 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.473774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.473776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.473981 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.474096 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.474680 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.474793 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 04:39:07.476275 14404 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 04:39:07.479238 14170 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [term 1 FOLLOWER]: Refusing update from remote peer b8b4e630043949719ec55667d328d310: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:39:07.480576 14329 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:39:07.481129 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:07.464311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.12.15.254:46295)
0806 04:39:07.464455 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.464460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.468844 (+ 4384us) server_negotiation.cc:407] Connection header received
0806 04:39:07.468961 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.468964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.469029 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.469102 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.472375 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.473121 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.474211 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:07.474534 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:07.474611 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:07.477548 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:07.477552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:07.477555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:07.477592 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:07.479189 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.479674 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.479677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.479679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.479765 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:07.480047 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:07.480050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:07.480052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:07.480269 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:07.480440 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:07.480847 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 04:39:07.480973 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 04:39:07.485772 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8b4e630043949719ec55667d328d310. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.485908 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.480458 14282 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [term 1 FOLLOWER]: Refusing update from remote peer b8b4e630043949719ec55667d328d310: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:39:07.486788 14373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 04:39:07.487133 14404 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:39:07.490911 14387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8b4e630043949719ec55667d328d310. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.491027 14387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.497768 14329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.497907 14329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:07.498633 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.498768 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.508472 14373 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.508642 14373 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:07.509766 14384 catalog_manager.cc:1347] Generated new cluster ID: cda24ad7296d498ba9aa67ba96b10e35
I20260806 04:39:07.509830 14384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:39:07.509848 14387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8b4e630043949719ec55667d328d310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b4e630043949719ec55667d328d310" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 46295 } } peers { permanent_uuid: "dc4a5f25e7af4e909f505de9f75125ae" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 42751 } } peers { permanent_uuid: "805b874c61624f24baca92429580f852" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 41741 } } }
I20260806 04:39:07.509948 14387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:07.510665 14410 mvcc.cc:205] Tried to move back new op lower bound from 7315419740041490432 to 7315419739963781120. Current Snapshot: MvccSnapshot[applied={T|T < 7315419740041490432}]
I20260806 04:39:07.533269 14384 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:39:07.534660 14384 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:39:07.550432 14384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8b4e630043949719ec55667d328d310: Generated new TSK 0
I20260806 04:39:07.551384 14384 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:39:07.586563 14112 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.583151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.12.15.254:46295)
0806 04:39:07.583286 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.583291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.585403 (+ 2112us) server_negotiation.cc:407] Connection header received
0806 04:39:07.585607 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.585611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.585666 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.585719 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.586415 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33672: BlockingRecv error: recv got EOF from 127.0.0.1:33672 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 04:39:07.587898 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.583679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.12.15.253:42751)
0806 04:39:07.583793 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.583797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.587058 (+ 3261us) server_negotiation.cc:407] Connection header received
0806 04:39:07.587225 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.587228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.587274 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.587321 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.587783 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59508: BlockingRecv error: recv got EOF from 127.0.0.1:59508 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 04:39:07.589207 14322 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.584700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.12.15.252:41741)
0806 04:39:07.584827 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.584831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.588374 (+ 3543us) server_negotiation.cc:407] Connection header received
0806 04:39:07.588537 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.588540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.588588 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.588635 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.589092 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60046: BlockingRecv error: recv got EOF from 127.0.0.1:60046 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 04:39:07.594028 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.592454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.12.15.253:42751)
0806 04:39:07.592557 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.592561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.593041 (+ 480us) server_negotiation.cc:407] Connection header received
0806 04:39:07.593266 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.593269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.593347 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.593398 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.593912 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59514: BlockingRecv error: recv got EOF from 127.0.0.1:59514 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 04:39:07.599020 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.597560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.12.15.253:42751)
0806 04:39:07.597657 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.597661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.598153 (+ 492us) server_negotiation.cc:407] Connection header received
0806 04:39:07.598335 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.598339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.598389 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.598437 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.598914 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59522: BlockingRecv error: recv got EOF from 127.0.0.1:59522 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 04:39:07.605044 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.603330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.12.15.253:42751)
0806 04:39:07.603434 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.603438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.604022 (+ 584us) server_negotiation.cc:407] Connection header received
0806 04:39:07.604215 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.604219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.604272 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.604323 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.604943 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59528: BlockingRecv error: recv got EOF from 127.0.0.1:59528 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 04:39:07.613233 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.611475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.12.15.253:42751)
0806 04:39:07.611623 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.611628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.612193 (+ 565us) server_negotiation.cc:407] Connection header received
0806 04:39:07.612407 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.612410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.612478 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.612533 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.613122 (+ 589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59530: BlockingRecv error: recv got EOF from 127.0.0.1:59530 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 04:39:07.626019 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.623820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.12.15.253:42751)
0806 04:39:07.623940 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.623945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.624678 (+ 733us) server_negotiation.cc:407] Connection header received
0806 04:39:07.624974 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.624978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.625042 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.625117 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.625875 (+ 758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59544: BlockingRecv error: recv got EOF from 127.0.0.1:59544 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 04:39:07.646060 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.644736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.12.15.253:42751)
0806 04:39:07.644863 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.644868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.645093 (+ 225us) server_negotiation.cc:407] Connection header received
0806 04:39:07.645275 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.645278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.645339 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.645421 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.645926 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59552: BlockingRecv error: recv got EOF from 127.0.0.1:59552 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 04:39:07.681442 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.680307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.12.15.253:42751)
0806 04:39:07.680457 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.680463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.680627 (+ 164us) server_negotiation.cc:407] Connection header received
0806 04:39:07.680745 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.680749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.680804 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.680895 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.681313 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59566: BlockingRecv error: recv got EOF from 127.0.0.1:59566 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 04:39:07.748750 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.747519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.12.15.253:42751)
0806 04:39:07.747669 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.747674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.747841 (+ 167us) server_negotiation.cc:407] Connection header received
0806 04:39:07.747993 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.747996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.748043 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.748115 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.748648 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59570: BlockingRecv error: recv got EOF from 127.0.0.1:59570 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 04:39:07.880146 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:07.879026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.12.15.253:42751)
0806 04:39:07.879155 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:07.879160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:07.879247 (+ 87us) server_negotiation.cc:407] Connection header received
0806 04:39:07.879370 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:07.879372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:07.879423 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:07.879512 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:07.880017 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59582: BlockingRecv error: recv got EOF from 127.0.0.1:59582 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 04:39:08.139935 14216 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:08.138674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59594 (local address 127.12.15.253:42751)
0806 04:39:08.138804 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:08.138809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:08.138967 (+ 158us) server_negotiation.cc:407] Connection header received
0806 04:39:08.139118 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:08.139121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:08.139170 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:08.139239 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:08.139804 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59594: BlockingRecv error: recv got EOF from 127.0.0.1:59594 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:39:08.448818 14367 catalog_manager.cc:1259] Loaded cluster ID: cda24ad7296d498ba9aa67ba96b10e35
I20260806 04:39:08.448909 14367 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: loading cluster ID for follower catalog manager: success
I20260806 04:39:08.450572 14367 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: acquiring CA information for follower catalog manager: success
I20260806 04:39:08.451699 14367 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc4a5f25e7af4e909f505de9f75125ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:39:08.488597 14404 catalog_manager.cc:1259] Loaded cluster ID: cda24ad7296d498ba9aa67ba96b10e35
I20260806 04:39:08.488667 14404 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: loading cluster ID for follower catalog manager: success
I20260806 04:39:08.490271 14404 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: acquiring CA information for follower catalog manager: success
I20260806 04:39:08.491403 14404 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 805b874c61624f24baca92429580f852: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:39:08.658960 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:08.654630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59598 (local address 127.12.15.253:42751)
0806 04:39:08.654977 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:08.654982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:08.655003 (+ 21us) server_negotiation.cc:407] Connection header received
0806 04:39:08.655086 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:08.655089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:08.655223 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:08.655327 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:08.655654 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.656317 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.657317 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.657566 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.657666 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:08.657995 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:08.658536 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:08.658639 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:08.658705 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 04:39:08.658775 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":140,"threads_started":1}
I20260806 04:39:08.665740 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:08.662183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.12.15.252:41741)
0806 04:39:08.662384 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:08.662388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:08.662411 (+ 23us) server_negotiation.cc:407] Connection header received
0806 04:39:08.662541 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:08.662544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:08.662662 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:08.662759 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:08.662981 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.663478 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.664284 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.664563 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.664659 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:08.664932 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:08.665359 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:08.665451 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:08.665559 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 04:39:08.665609 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":57,"threads_started":1}
I20260806 04:39:08.672418 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:08.668609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59614 (local address 127.12.15.253:42751)
0806 04:39:08.668735 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:08.668739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:08.669018 (+ 279us) server_negotiation.cc:407] Connection header received
0806 04:39:08.669253 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:08.669257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:08.669308 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:08.669373 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:08.669610 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.670298 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.671293 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.671565 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.671671 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:08.671916 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:08.672008 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:08.672071 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:08.672192 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 04:39:08.672249 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:39:08.673080 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:08.669039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.12.15.252:41741)
0806 04:39:08.669154 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:08.669157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:08.669524 (+ 367us) server_negotiation.cc:407] Connection header received
0806 04:39:08.669586 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:08.669589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:08.669636 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:08.669694 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:08.670020 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.670716 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.671729 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.672033 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.672132 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:08.672475 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:08.672546 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:08.672606 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:08.672896 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 04:39:08.672950 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:39:08.673197 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:08.668665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.12.15.254:46295)
0806 04:39:08.668904 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:08.668908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:08.669154 (+ 246us) server_negotiation.cc:407] Connection header received
0806 04:39:08.669323 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:08.669327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:08.669499 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:08.669608 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:08.669830 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.670415 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.671465 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:08.671806 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:08.671901 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:08.672242 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:08.672658 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:08.672751 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:08.673006 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 04:39:08.673066 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":61,"threads_started":1}
I20260806 04:39:08.676774 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14004
I20260806 04:39:08.696262 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14117
I20260806 04:39:08.708033 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14223
2026-08-06T04:39:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3221 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 04:39:08 dist-test-slave-xt4k krb5kdc[14528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 04:39:08 dist-test-slave-xt4k krb5kdc[14528](info): set up 2 sockets
Aug 06 04:39:08 dist-test-slave-xt4k krb5kdc[14528](info): commencing operation
krb5kdc: starting...
W20260806 04:39:09.856272 12351 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/test-user.keytab.
Aug 06 04:39:09 dist-test-slave-xt4k krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T04:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T04:39:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
Entry for principal kudu/127.12.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab.
I20260806 04:39:09.984993 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:32973
--webserver_interface=127.12.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:32973,127.12.15.253:38337,127.12.15.252:46457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 04:39:10.101701 14549 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 04:39:10.101991 14549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:10.102033 14549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:10.102094 14549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 04:39:10.106033 14549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:10.106086 14549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:10.106112 14549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:10.106134 14549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:10.106163 14549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:10.111289 14549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:32973,127.12.15.253:38337,127.12.15.252:46457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.254.keytab
--principal=kudu/127.12.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.254:32973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:10.112289 14549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:10.113297 14549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 04:39:10.121539 14561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:10.121616 14556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:10.121616 14555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:10.121922 14549 server_base.cc:1034] running on GCE node
I20260806 04:39:10.122483 14549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:10.123549 14549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:10.124699 14549 hybrid_clock.cc:648] HybridClock initialized: now 1785991150124684 us; error 29 us; skew 500 ppm
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:10.127372 14549 init.cc:377] Logged in from keytab as kudu/127.12.15.254@KRBTEST.COM (short username kudu)
I20260806 04:39:10.128253 14549 webserver.cc:460] Webserver started at http://127.12.15.254:41951/ using document root <none> and password file <none>
I20260806 04:39:10.128818 14549 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:10.128877 14549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:10.129045 14549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:10.131037 14549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data/instance:
uuid: "f09963c302724f7bbfdad9c446c1d621"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "2a4e7cda21b05798203cfb0b0dc4dac6"
server_key_iv: "60a4c9aea14ef023bc91df9148bec9a8"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.131557 14549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal/instance:
uuid: "f09963c302724f7bbfdad9c446c1d621"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "2a4e7cda21b05798203cfb0b0dc4dac6"
server_key_iv: "60a4c9aea14ef023bc91df9148bec9a8"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.135221 14549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 04:39:10.137643 14571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:10.138968 14549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 04:39:10.139081 14549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal
uuid: "f09963c302724f7bbfdad9c446c1d621"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "2a4e7cda21b05798203cfb0b0dc4dac6"
server_key_iv: "60a4c9aea14ef023bc91df9148bec9a8"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.139173 14549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:10.150400 14549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:10.153080 14549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:10.153231 14549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:10.159960 14549 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.254:32973
I20260806 04:39:10.160226 14649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.254:32973 every 8 connection(s)
I20260806 04:39:10.160957 14549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/data/info.pb
I20260806 04:39:10.162688 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14549
I20260806 04:39:10.162784 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-0/wal/instance
I20260806 04:39:10.163053 12351 external_mini_cluster.cc:1428] Setting key 006456f00b9a7db20a16d12127eef0ec
I20260806 04:39:10.163883 14650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:10.167614 14650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.15.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.253@KRBTEST.COM" created.
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:39:10.181923 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.169830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:32973 (local address 127.0.0.1:49940)
0806 04:39:10.170309 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.170361 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.171202 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.171911 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.171916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.172366 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.172647 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.172661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.173788 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.173791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.174643 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.174654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.175264 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.175268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.175375 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.175927 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.175944 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.177399 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.179120 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.179128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.179138 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.179351 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.179635 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.179638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.179640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.179744 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.180298 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.180304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.180458 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.180615 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.180987 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":88,"thread_start_us":104,"threads_started":1}
I20260806 04:39:10.181923 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.169831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.12.15.254:32973)
0806 04:39:10.170391 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.170402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.170456 (+ 54us) server_negotiation.cc:407] Connection header received
0806 04:39:10.171486 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.171490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.171683 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.171849 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.172777 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.173649 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.174759 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.175121 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.175262 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.177531 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.177536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.177548 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.177570 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.179015 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.179448 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.179453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.179454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.179552 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.179823 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.179826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.179828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.180194 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.180310 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.180689 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.180897 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":162,"threads_started":1}
I20260806 04:39:10.184371 14650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:10.185082 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.184812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58216)
0806 04:39:10.184930 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.185009 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 04:39:10.185261 14574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
I20260806 04:39:10.186074 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.185875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58218)
0806 04:39:10.185973 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.186018 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 04:39:10.186200 14650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:38337: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab.
Entry for principal kudu/127.12.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab.
I20260806 04:39:10.190901 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:38337
--webserver_interface=127.12.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab
--principal=kudu/127.12.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:32973,127.12.15.253:38337,127.12.15.252:46457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:39:10.235482 14650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } attempt: 1
I20260806 04:39:10.236407 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.236018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58226)
0806 04:39:10.236201 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.236281 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 04:39:10.237169 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.236945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58240)
0806 04:39:10.237046 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.237102 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 04:39:10.237321 14650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:38337: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
W20260806 04:39:10.304003 14661 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 04:39:10.304320 14661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:10.304384 14661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:10.304448 14661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 04:39:10.305483 14650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } attempt: 2
I20260806 04:39:10.306327 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.305989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58248)
0806 04:39:10.306165 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.306235 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 04:39:10.307016 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.306809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58262)
0806 04:39:10.306902 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.306950 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 04:39:10.307145 14650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.253:38337: Network error: Client connection negotiation failed: client connection to 127.12.15.253:38337: connect: Connection refused (error 111)
W20260806 04:39:10.308853 14661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:10.308904 14661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:10.308923 14661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:10.308938 14661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:10.308951 14661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:10.314332 14661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:32973,127.12.15.253:38337,127.12.15.252:46457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.253.keytab
--principal=kudu/127.12.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.253:38337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:10.315415 14661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:10.316486 14661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:39:10.325058 14661 server_base.cc:1034] running on GCE node
W20260806 04:39:10.325001 14669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:10.325112 14674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:10.325007 14672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:10.325801 14661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:10.326928 14661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:10.328095 14661 hybrid_clock.cc:648] HybridClock initialized: now 1785991150328082 us; error 28 us; skew 500 ppm
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:10.330899 14661 init.cc:377] Logged in from keytab as kudu/127.12.15.253@KRBTEST.COM (short username kudu)
I20260806 04:39:10.331878 14661 webserver.cc:460] Webserver started at http://127.12.15.253:40661/ using document root <none> and password file <none>
I20260806 04:39:10.332501 14661 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:10.332585 14661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:10.332825 14661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:10.334932 14661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data/instance:
uuid: "be453b829cfa4103b27774fb4b554cca"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "8debcb82a3683804c5b4158cbed44c76"
server_key_iv: "8deaeb14700ab0de9ae65d3965fab97b"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.335506 14661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal/instance:
uuid: "be453b829cfa4103b27774fb4b554cca"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "8debcb82a3683804c5b4158cbed44c76"
server_key_iv: "8deaeb14700ab0de9ae65d3965fab97b"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.339429 14661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 04:39:10.342201 14685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:10.343659 14661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 04:39:10.343794 14661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal
uuid: "be453b829cfa4103b27774fb4b554cca"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "8debcb82a3683804c5b4158cbed44c76"
server_key_iv: "8deaeb14700ab0de9ae65d3965fab97b"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.343899 14661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:10.364595 14661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:10.367908 14661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:10.368098 14661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:10.375231 14661 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.253:38337
I20260806 04:39:10.375550 14761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.253:38337 every 8 connection(s)
I20260806 04:39:10.376247 14661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/data/info.pb
I20260806 04:39:10.377875 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14661
I20260806 04:39:10.378005 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-1/wal/instance
I20260806 04:39:10.378340 12351 external_mini_cluster.cc:1428] Setting key a7c1e1a88942122eef9e3fa694fe665c
I20260806 04:39:10.379673 14762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:10.383790 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.15.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.252@KRBTEST.COM" created.
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:39:10.398216 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.385762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.12.15.254:32973)
0806 04:39:10.385939 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.385945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.386771 (+ 826us) server_negotiation.cc:407] Connection header received
0806 04:39:10.387737 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.387741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.387795 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.387885 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.389207 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.389858 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.391365 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.391632 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.391727 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.394402 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.394405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.394407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.394436 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.395848 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.396514 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.396517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.396519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.396624 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.397001 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.397003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.397004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.397185 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.397258 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.397962 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.398080 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 04:39:10.398384 14650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } attempt: 3
I20260806 04:39:10.399333 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.386094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:32973 (local address 127.0.0.1:49950)
0806 04:39:10.386601 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.386654 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.387523 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.388058 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.388078 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.388516 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.389006 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.389020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.390008 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.390011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.391226 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.391238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.391792 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.391796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.392013 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.392645 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.392662 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.394190 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.395978 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.395986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.396005 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.396413 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.396713 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.396716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.396718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.396915 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.397283 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.397290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.397657 (+ 367us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.397876 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.398233 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260806 04:39:10.400979 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab.
Entry for principal kudu/127.12.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab.
I20260806 04:39:10.408768 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.398800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58274)
0806 04:39:10.398953 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.398968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.399052 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.399754 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.399757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.399893 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.399987 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.399993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.401157 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.401161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.401995 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.402008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.402623 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.402627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.402829 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.403594 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.403607 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.405038 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.407026 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.407030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.407032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.407242 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.407589 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.407592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.407595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.407693 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.408215 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.408218 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.408328 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.408451 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.408596 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 04:39:10.408936 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.398984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58274 (local address 127.12.15.253:38337)
0806 04:39:10.399347 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.399356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.399375 (+ 19us) server_negotiation.cc:407] Connection header received
0806 04:39:10.399412 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.399418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.399622 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.399756 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.400097 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.401032 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.402103 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.402516 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.402611 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.405166 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.405175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.405187 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.405208 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.406928 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.407358 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.407364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.407366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.407447 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.407805 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.407809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.407811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.408114 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.408280 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.408563 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.408766 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":168,"threads_started":1}
I20260806 04:39:10.408950 12351 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5_1OQt/build/debug/bin/kudu
/tmp/dist-test-task5_1OQt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:46457
--webserver_interface=127.12.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab
--principal=kudu/127.12.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.212:34099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.254:32973,127.12.15.253:38337,127.12.15.252:46457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 04:39:10.410082 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.401464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58290)
0806 04:39:10.401554 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.401567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.401648 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.402024 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.402027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.402176 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.402254 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.402260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.402974 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.402977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.403583 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.403593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.404057 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.404061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.404144 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.404577 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.404589 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.405898 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.408523 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.408526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.408527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.408816 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.409132 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.409134 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.409136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.409211 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.409574 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.409577 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.409669 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.409774 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.409902 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 04:39:10.410195 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.401560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58290 (local address 127.12.15.253:38337)
0806 04:39:10.401845 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.401851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.401862 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:39:10.401888 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.401890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.401921 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.401995 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.402351 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.402881 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.403683 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.403958 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.404046 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.406254 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.406257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.406259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.406281 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.408407 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.408919 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.408922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.408924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.409027 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.409287 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.409289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.409291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.409455 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.409543 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.409832 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.409976 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":234,"thread_start_us":107,"threads_started":1}
I20260806 04:39:10.412107 14650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:10.412478 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:10.412878 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.412622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42922)
0806 04:39:10.412742 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.412799 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 04:39:10.413116 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.412842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42930)
0806 04:39:10.412961 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.413042 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 04:39:10.413319 14689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
I20260806 04:39:10.414114 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.413811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42942)
0806 04:39:10.413997 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.414046 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 04:39:10.414182 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.413943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42944)
0806 04:39:10.414054 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.414117 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 04:39:10.414343 14650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:46457: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
W20260806 04:39:10.414304 14762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:46457: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
I20260806 04:39:10.445488 14650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } attempt: 1
I20260806 04:39:10.446228 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.445953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42958)
0806 04:39:10.446096 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.446154 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 04:39:10.446856 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.446660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42970)
0806 04:39:10.446753 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.446799 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 04:39:10.446964 14650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:46457: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
I20260806 04:39:10.463548 14762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } attempt: 1
I20260806 04:39:10.464321 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.464050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42972)
0806 04:39:10.464189 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.464246 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 04:39:10.465018 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.464824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:42986)
0806 04:39:10.464919 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.464962 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 04:39:10.465145 14762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:46457: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
W20260806 04:39:10.521833 14770 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 04:39:10.522100 14770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 04:39:10.522135 14770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 04:39:10.522197 14770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 04:39:10.522114 14650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } attempt: 2
I20260806 04:39:10.522850 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.522579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43000)
0806 04:39:10.522719 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.522776 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 04:39:10.523471 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.523274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43004)
0806 04:39:10.523370 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.523414 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 04:39:10.523613 14650 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:46457: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
W20260806 04:39:10.526232 14770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 04:39:10.526283 14770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 04:39:10.526309 14770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 04:39:10.526331 14770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 04:39:10.526361 14770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 04:39:10.531384 14770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.212:34099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.254:32973,127.12.15.253:38337,127.12.15.252:46457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/krb5kdc/kudu_127.12.15.252.keytab
--principal=kudu/127.12.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.252:46457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 03:57:55 UTC on ccycloud.quasar-jhkjat.root.comops.site
build id 4569616
I20260806 04:39:10.532446 14770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 04:39:10.533308 14762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } attempt: 2
I20260806 04:39:10.533471 14770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 04:39:10.534091 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.533807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43012)
0806 04:39:10.533962 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.534017 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 04:39:10.534760 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.534552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43028)
0806 04:39:10.534659 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.534704 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 04:39:10.534901 14762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.252:46457: Network error: Client connection negotiation failed: client connection to 127.12.15.252:46457: connect: Connection refused (error 111)
W20260806 04:39:10.541823 14779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 04:39:10.541831 14784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:10.541985 14770 server_base.cc:1034] running on GCE node
W20260806 04:39:10.542163 14780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 04:39:10.542757 14770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 04:39:10.543908 14770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 04:39:10.545065 14770 hybrid_clock.cc:648] HybridClock initialized: now 1785991150545042 us; error 40 us; skew 500 ppm
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 04:39:10.547741 14770 init.cc:377] Logged in from keytab as kudu/127.12.15.252@KRBTEST.COM (short username kudu)
I20260806 04:39:10.548732 14770 webserver.cc:460] Webserver started at http://127.12.15.252:41785/ using document root <none> and password file <none>
I20260806 04:39:10.549276 14770 fs_manager.cc:359] Metadata directory not provided
I20260806 04:39:10.549337 14770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 04:39:10.549546 14770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 04:39:10.551573 14770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data/instance:
uuid: "0762967d70b14be7979291d575819cdf"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "7068338be1f20d4f36dc5a8a8c23a8e1"
server_key_iv: "bdcc317ef13b2d54ea509cc0f956cd25"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.552129 14770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal/instance:
uuid: "0762967d70b14be7979291d575819cdf"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "7068338be1f20d4f36dc5a8a8c23a8e1"
server_key_iv: "bdcc317ef13b2d54ea509cc0f956cd25"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.555791 14770 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 04:39:10.558338 14793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 04:39:10.559527 14770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 04:39:10.559644 14770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data,/tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal
uuid: "0762967d70b14be7979291d575819cdf"
format_stamp: "Formatted at 2026-08-06 04:39:10 on dist-test-slave-xt4k"
server_key: "7068338be1f20d4f36dc5a8a8c23a8e1"
server_key_iv: "bdcc317ef13b2d54ea509cc0f956cd25"
server_key_version: "encryptionkey@0"
I20260806 04:39:10.559751 14770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 04:39:10.602102 14770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 04:39:10.605016 14770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 04:39:10.605206 14770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 04:39:10.612025 14770 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.252:46457
I20260806 04:39:10.612285 14865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.252:46457 every 8 connection(s)
I20260806 04:39:10.613025 14770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/data/info.pb
I20260806 04:39:10.615727 14866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 04:39:10.617228 12351 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5_1OQt/build/debug/bin/kudu as pid 14770
I20260806 04:39:10.617360 12351 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5_1OQt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785990859032933-12351-0/minicluster-data/master-2/wal/instance
I20260806 04:39:10.617707 12351 external_mini_cluster.cc:1428] Setting key 5a4219a1cbd827651cf670a0a60982cb
I20260806 04:39:10.620011 14866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:10.622759 14650 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } attempt: 3
I20260806 04:39:10.626049 14762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } attempt: 3
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.254@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
I20260806 04:39:10.632735 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.623165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43038)
0806 04:39:10.623560 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.623583 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.623661 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.623891 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.623902 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.624073 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.624152 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.624157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.624962 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.624965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.625657 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.625669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.626257 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.626261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.626343 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.626832 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.626848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.628650 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.631148 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.631153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.631382 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.631724 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.631729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.631815 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.632232 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.632235 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.632326 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.632434 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.632551 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327}
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.253@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
I20260806 04:39:10.633563 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.623220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43038 (local address 127.12.15.252:46457)
0806 04:39:10.623459 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.623466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.623649 (+ 183us) server_negotiation.cc:407] Connection header received
0806 04:39:10.623735 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.623738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.623779 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.623863 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.624251 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.624838 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.625794 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.626122 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.626541 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.628820 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.628824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.628827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.628854 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.631030 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631506 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.631509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.631510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.631591 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631922 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.631925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.631927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.632124 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.633171 (+ 1047us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.633246 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.633353 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":180,"thread_start_us":83,"threads_started":1}
I20260806 04:39:10.633579 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.618959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.12.15.252:46457)
0806 04:39:10.619344 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.619359 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.619393 (+ 34us) server_negotiation.cc:407] Connection header received
0806 04:39:10.620020 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.620036 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.620421 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.620697 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.621587 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.623042 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.624039 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.624753 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.624912 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.628430 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.628437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.628462 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.628485 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.630870 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631388 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.631393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.631395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.631482 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631783 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.631787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.631788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.632120 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.632903 (+ 783us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.633013 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.633181 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"spinlock_wait_cycles":7040,"thread_start_us":100,"threads_started":1}
I20260806 04:39:10.634123 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.621936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.12.15.254:32973)
0806 04:39:10.622040 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.622043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.622375 (+ 332us) server_negotiation.cc:407] Connection header received
0806 04:39:10.622520 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.622522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.622563 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.622625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.623334 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.623991 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.625168 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.625459 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.625556 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.628324 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.628327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.628330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.628380 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.630055 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.630730 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.630734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.630736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.630813 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.631171 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.631175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.631176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.631493 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.631577 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.633864 (+ 2287us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.633975 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 04:39:10.633563 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.621909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.254:32973 (local address 127.0.0.1:49954)
0806 04:39:10.622264 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.622301 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.622424 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.622685 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.622691 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.623087 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.623243 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.623250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.624099 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.624102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.625051 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.625059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.625618 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.625623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.625725 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.626362 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.626383 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.628162 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.630197 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.630205 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.630215 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.630640 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.630897 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.630900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.630902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.631075 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.631627 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.631634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.631960 (+ 326us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.632164 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.632523 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":53,"thread_start_us":138,"threads_started":1}
I20260806 04:39:10.636694 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.626478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43040)
0806 04:39:10.627121 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.627136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.627220 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.627611 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.627614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.627808 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.627921 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.627929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.630188 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.630192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.631000 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.631011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.631600 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.631604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.631693 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.632128 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.632142 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.633500 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.635202 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.635205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.635207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.635419 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.635715 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.635718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.635720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.635813 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.636176 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.636179 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.636271 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.636397 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.636549 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 04:39:10.636709 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.627119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.12.15.252:46457)
0806 04:39:10.627414 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.627422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.627433 (+ 11us) server_negotiation.cc:407] Connection header received
0806 04:39:10.627460 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.627462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.627493 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.627564 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.628061 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.629852 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.631141 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.631451 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.633642 (+ 2191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.633739 (+ 97us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.633742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.633744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.633759 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.635097 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.635520 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.635523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.635525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.635610 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.635920 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.635923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.635924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.636081 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.636162 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.636449 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.636546 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":88,"threads_started":1}
I20260806 04:39:10.637422 14866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } has no permanent_uuid. Determining permanent_uuid...
I20260806 04:39:10.638489 14650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: Bootstrap starting.
I20260806 04:39:10.640110 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: Bootstrap starting.
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.254@KRBTEST.COM
I20260806 04:39:10.641690 14650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:10.642617 14762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:10.642642 14650 log.cc:824] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:10.644138 14762 log.cc:824] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:10.645054 14650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: No bootstrap required, opened a new log
I20260806 04:39:10.645972 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.637041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.12.15.254:32973)
0806 04:39:10.637611 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.637615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.637628 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:39:10.637658 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.637661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.637700 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.637772 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.638430 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.638945 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.639686 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.639895 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.639977 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.642217 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.642221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.642223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.642244 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.643757 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.644234 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.644237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.644238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.644357 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.644687 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.644690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.644692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.644891 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.645653 (+ 762us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.645726 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.645835 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496}
I20260806 04:39:10.646124 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: No bootstrap required, opened a new log
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
I20260806 04:39:10.649325 14650 raft_consensus.cc:348] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.649544 14650 raft_consensus.cc:729] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09963c302724f7bbfdad9c446c1d621, State: Initialized, Role: FOLLOWER
I20260806 04:39:10.649801 14650 consensus_queue.cc:260] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.649758 14762 raft_consensus.cc:348] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.649983 14762 raft_consensus.cc:729] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be453b829cfa4103b27774fb4b554cca, State: Initialized, Role: FOLLOWER
I20260806 04:39:10.650200 14883 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.650318 14883 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.650279 14762 consensus_queue.cc:260] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.650771 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.650904 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.651083 14650 sys_catalog.cc:564] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:10.651575 14762 sys_catalog.cc:564] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:10.651942 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.637886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.253:38337 (local address 127.0.0.1:58294)
0806 04:39:10.643072 (+ 5186us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.643086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.643173 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.643469 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.643472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.643643 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.643766 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.643773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.644762 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.644765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.645534 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.645546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.646260 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.646263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.646358 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.647036 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.647050 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.648426 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.650272 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.650275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.650277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.650514 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.650831 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.650835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.650837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.650927 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.651410 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.651413 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.651510 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.651642 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.651780 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5113}
I20260806 04:39:10.651973 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.640613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.12.15.253:38337)
0806 04:39:10.640755 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.640760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.643141 (+ 2381us) server_negotiation.cc:407] Connection header received
0806 04:39:10.643241 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.643244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.643292 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.643357 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.643900 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.644620 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.645671 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.646036 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.646111 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.648553 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.648557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.648560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.648584 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.650148 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.650643 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.650645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.650647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.650730 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.651053 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.651056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.651058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.651295 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.651395 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.651704 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.651820 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 04:39:10.653504 14866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } has no permanent_uuid. Determining permanent_uuid...
W20260806 04:39:10.654636 14578 tablet.cc:2369] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.253@KRBTEST.COM
W20260806 04:39:10.658687 14914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:39:10.659632 14914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 04:39:10 dist-test-slave-xt4k krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785991150, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.252@KRBTEST.COM for kudu/127.12.15.252@KRBTEST.COM
I20260806 04:39:10.662598 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.652416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.12.15.253:38337)
0806 04:39:10.652528 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.652532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.652544 (+ 12us) server_negotiation.cc:407] Connection header received
0806 04:39:10.652574 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.652577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.652618 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.652685 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.653460 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.654104 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.655100 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.655388 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.655481 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.658371 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.658374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.658377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.658402 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.660207 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.660717 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.660720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.660722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.660808 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.661104 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.661108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.661110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.661322 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.661513 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.662091 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.662202 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 04:39:10.659610 14661 master_runner.cc:186] Error getting master registration for 127.12.15.252:46457: OK, instance_id { permanent_uuid: "0762967d70b14be7979291d575819cdf" instance_seqno: 1785991150610195 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 04:39:10.662772 14661 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 04:39:10.664765 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.654008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.252:46457 (local address 127.0.0.1:43044)
0806 04:39:10.654244 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 04:39:10.654256 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 04:39:10.654329 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 04:39:10.654756 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 04:39:10.654758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 04:39:10.654910 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 04:39:10.654976 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.654982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.657475 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.657479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.658234 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 04:39:10.658244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.659199 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.659202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 04:39:10.659280 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.659923 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 04:39:10.659937 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 04:39:10.661451 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 04:39:10.663395 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.663398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.663400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.663629 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.663865 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 04:39:10.663868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 04:39:10.663870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 04:39:10.663942 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 04:39:10.664259 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 04:39:10.664261 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 04:39:10.664379 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 04:39:10.664480 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 04:39:10.664613 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 04:39:10.665251 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.654228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.12.15.252:46457)
0806 04:39:10.654516 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.654520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.654533 (+ 13us) server_negotiation.cc:407] Connection header received
0806 04:39:10.654564 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.654567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.654605 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.654674 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.655394 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.656105 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.658613 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.658971 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.659094 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.661569 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.661573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.661575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.661596 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.663088 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.663715 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.663718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.663720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.663792 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.664015 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.664018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.664020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.664177 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.664258 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.664532 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.665143 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":495,"server-negotiator.queue_time_us":215}
I20260806 04:39:10.667517 14866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: Bootstrap starting.
I20260806 04:39:10.664731 14549 master_runner.cc:186] Error getting master registration for 127.12.15.252:46457: OK, instance_id { permanent_uuid: "0762967d70b14be7979291d575819cdf" instance_seqno: 1785991150610195 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 04:39:10.670269 14866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: Neither blocks nor log segments found. Creating new log.
I20260806 04:39:10.670497 12351 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 04:39:10.671201 14866 log.cc:824] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: Log is configured to *not* fsync() on all Append() calls
I20260806 04:39:10.671767 14549 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 04:39:10.671775 14608 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f09963c302724f7bbfdad9c446c1d621"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49956
I20260806 04:39:10.672013 14608 raft_consensus.cc:482] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 04:39:10.672127 14608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:10.673192 14608 raft_consensus.cc:504] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.673987 14608 leader_election.cc:288] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [CANDIDATE]: Term 1 election: Requested vote from peers be453b829cfa4103b27774fb4b554cca (127.12.15.253:38337), 0762967d70b14be7979291d575819cdf (127.12.15.252:46457)
I20260806 04:39:10.674604 14719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f09963c302724f7bbfdad9c446c1d621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "be453b829cfa4103b27774fb4b554cca"
I20260806 04:39:10.674850 14719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [term 0 FOLLOWER]: Advancing to term 1
W20260806 04:39:10.671604 14898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:39:10.675930 14898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:39:10.676194 14719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f09963c302724f7bbfdad9c446c1d621 in term 1.
I20260806 04:39:10.676604 14574 leader_election.cc:302] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be453b829cfa4103b27774fb4b554cca, f09963c302724f7bbfdad9c446c1d621; no voters:
I20260806 04:39:10.676896 14883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 04:39:10.677114 14883 raft_consensus.cc:686] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [term 1 LEADER]: Becoming Leader. State: Replica: f09963c302724f7bbfdad9c446c1d621, State: Running, Role: LEADER
I20260806 04:39:10.677485 14883 consensus_queue.cc:237] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.679484 14883 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f09963c302724f7bbfdad9c446c1d621. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.679631 14883 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:10.683336 14866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: No bootstrap required, opened a new log
I20260806 04:39:10.692836 14866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.693440 14866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0762967d70b14be7979291d575819cdf, State: Initialized, Role: FOLLOWER
I20260806 04:39:10.694057 14866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } }
I20260806 04:39:10.696209 14932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 04:39:10.696838 14827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f09963c302724f7bbfdad9c446c1d621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0762967d70b14be7979291d575819cdf"
I20260806 04:39:10.697036 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.676228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.12.15.253:38337)
0806 04:39:10.676576 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.676581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.678050 (+ 1469us) server_negotiation.cc:407] Connection header received
0806 04:39:10.678294 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.678297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.678349 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.678403 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.680839 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.681617 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.684557 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.684904 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.684990 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.688471 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.688475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.688479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.688513 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.690448 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.692101 (+ 1653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.692105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.692107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.692205 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.695209 (+ 3004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.695213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.695215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.695453 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.695566 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.696729 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.696875 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 04:39:10.697098 14827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 04:39:10.697845 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.677459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.12.15.254:32973)
0806 04:39:10.677568 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.677572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.688774 (+ 11202us) server_negotiation.cc:407] Connection header received
0806 04:39:10.688958 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.688962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.689016 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.689080 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.689909 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.690693 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.692211 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.692577 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.692661 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.694308 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.694312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.694315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.694342 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.695931 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.696442 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.696445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.696447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.696536 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.696823 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.696826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.696828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.697119 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.697226 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.697574 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.697693 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 04:39:10.698524 14827 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f09963c302724f7bbfdad9c446c1d621 in term 1.
I20260806 04:39:10.698979 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:10.677155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.12.15.252:46457)
0806 04:39:10.677270 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.677275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.678684 (+ 1409us) server_negotiation.cc:407] Connection header received
0806 04:39:10.679722 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.679726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.679782 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.679835 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.682767 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.683456 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.685555 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:10.685871 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:10.685949 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:10.688865 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 04:39:10.688868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 04:39:10.688871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 04:39:10.688901 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 04:39:10.691249 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.692328 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.692348 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.692350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.692438 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 04:39:10.694729 (+ 2291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 04:39:10.694733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 04:39:10.694734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 04:39:10.694965 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 04:39:10.695122 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:10.698704 (+ 3582us) server_negotiation.cc:299] Negotiation successful
0806 04:39:10.698839 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 04:39:10.699787 14866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 04:39:10.701191 14932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 04:39:10.705370 14945 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 04:39:10.705840 14945 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 04:39:10.705729 14933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.706048 14933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.717038 14827 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [term 1 FOLLOWER]: Refusing update from remote peer f09963c302724f7bbfdad9c446c1d621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:39:10.717494 14719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [term 1 FOLLOWER]: Refusing update from remote peer f09963c302724f7bbfdad9c446c1d621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 04:39:10.717730 14926 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:39:10.718066 14883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 04:39:10.722846 14933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f09963c302724f7bbfdad9c446c1d621. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.722980 14933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.726379 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f09963c302724f7bbfdad9c446c1d621. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.726496 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.733896 14883 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.734037 14883 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:10.734145 14950 mvcc.cc:205] Tried to move back new op lower bound from 7315419753330159616 to 7315419753178882048. Current Snapshot: MvccSnapshot[applied={T|T < 7315419753330159616}]
I20260806 04:39:10.735157 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.735267 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.737275 14957 mvcc.cc:205] Tried to move back new op lower bound from 7315419753330159616 to 7315419753178882048. Current Snapshot: MvccSnapshot[applied={T|T < 7315419753330159616}]
I20260806 04:39:10.739059 14926 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.739178 14926 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621 [sys.catalog]: This master's current role is: LEADER
I20260806 04:39:10.740058 14933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f09963c302724f7bbfdad9c446c1d621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f09963c302724f7bbfdad9c446c1d621" member_type: VOTER last_known_addr { host: "127.12.15.254" port: 32973 } } peers { permanent_uuid: "be453b829cfa4103b27774fb4b554cca" member_type: VOTER last_known_addr { host: "127.12.15.253" port: 38337 } } peers { permanent_uuid: "0762967d70b14be7979291d575819cdf" member_type: VOTER last_known_addr { host: "127.12.15.252" port: 46457 } } }
I20260806 04:39:10.740172 14933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 04:39:10.742147 14932 catalog_manager.cc:1347] Generated new cluster ID: 5c4028998e844b148c614d74727166cd
I20260806 04:39:10.742218 14932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 04:39:10.743925 14954 mvcc.cc:205] Tried to move back new op lower bound from 7315419753330159616 to 7315419753178882048. Current Snapshot: MvccSnapshot[applied={T|T < 7315419753330159616}]
I20260806 04:39:10.766680 14932 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 04:39:10.767976 14932 catalog_manager.cc:1504] Loading token signing keys...
I20260806 04:39:10.792734 14932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f09963c302724f7bbfdad9c446c1d621: Generated new TSK 0
I20260806 04:39:10.793677 14932 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 04:39:10.861214 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.857843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.12.15.253:38337)
0806 04:39:10.857976 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.857981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.859991 (+ 2010us) server_negotiation.cc:407] Connection header received
0806 04:39:10.860204 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.860208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.860270 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.860325 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.861071 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58314: BlockingRecv error: recv got EOF from 127.0.0.1:58314 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 04:39:10.862633 14654 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.858379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.12.15.254:32973)
0806 04:39:10.858496 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.858500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.861629 (+ 3129us) server_negotiation.cc:407] Connection header received
0806 04:39:10.861813 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.861816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.861868 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.861925 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.862499 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49978: BlockingRecv error: recv got EOF from 127.0.0.1:49978 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 04:39:10.864328 14880 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.859108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.12.15.252:46457)
0806 04:39:10.859220 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.859224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.863155 (+ 3931us) server_negotiation.cc:407] Connection header received
0806 04:39:10.863572 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.863587 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.863641 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.863693 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.864192 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43058: BlockingRecv error: recv got EOF from 127.0.0.1:43058 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 04:39:10.869095 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.867436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.12.15.253:38337)
0806 04:39:10.867546 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.867550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.868023 (+ 473us) server_negotiation.cc:407] Connection header received
0806 04:39:10.868257 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.868261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.868368 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.868427 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.868985 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58326: BlockingRecv error: recv got EOF from 127.0.0.1:58326 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 04:39:10.874116 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.873165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.12.15.253:38337)
0806 04:39:10.873286 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.873290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.873304 (+ 14us) server_negotiation.cc:407] Connection header received
0806 04:39:10.873430 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.873434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.873485 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.873587 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.874011 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58340: BlockingRecv error: recv got EOF from 127.0.0.1:58340 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 04:39:10.879272 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.877983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.12.15.253:38337)
0806 04:39:10.878104 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.878108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.878454 (+ 346us) server_negotiation.cc:407] Connection header received
0806 04:39:10.878572 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.878576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.878631 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.878715 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.879163 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58346: BlockingRecv error: recv got EOF from 127.0.0.1:58346 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 04:39:10.886782 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.885542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.12.15.253:38337)
0806 04:39:10.885714 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.885719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.885980 (+ 261us) server_negotiation.cc:407] Connection header received
0806 04:39:10.886109 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.886112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.886166 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.886245 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.886670 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58362: BlockingRecv error: recv got EOF from 127.0.0.1:58362 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260806 04:39:10.898207 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.897042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.12.15.253:38337)
0806 04:39:10.897162 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.897167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.897331 (+ 164us) server_negotiation.cc:407] Connection header received
0806 04:39:10.897489 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.897491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.897539 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.897608 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.898081 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58378: BlockingRecv error: recv got EOF from 127.0.0.1:58378 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 04:39:10.917271 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.916274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.12.15.253:38337)
0806 04:39:10.916418 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.916422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.916538 (+ 116us) server_negotiation.cc:407] Connection header received
0806 04:39:10.916649 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.916651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.916698 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.916786 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.917172 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58380: BlockingRecv error: recv got EOF from 127.0.0.1:58380 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 04:39:10.952622 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:10.951398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.12.15.253:38337)
0806 04:39:10.951524 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:10.951528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:10.951636 (+ 108us) server_negotiation.cc:407] Connection header received
0806 04:39:10.951841 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:10.951844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:10.951913 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:10.952037 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:10.952499 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58396: BlockingRecv error: recv got EOF from 127.0.0.1:58396 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 04:39:11.020139 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:11.018830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.12.15.253:38337)
0806 04:39:11.019019 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.019024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.019151 (+ 127us) server_negotiation.cc:407] Connection header received
0806 04:39:11.019321 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.019324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.019373 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.019441 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:11.020025 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58408: BlockingRecv error: recv got EOF from 127.0.0.1:58408 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 04:39:11.151382 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:11.150247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.12.15.253:38337)
0806 04:39:11.150384 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.150389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.150619 (+ 230us) server_negotiation.cc:407] Connection header received
0806 04:39:11.150724 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.150726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.150778 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.150855 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:11.151232 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58416: BlockingRecv error: recv got EOF from 127.0.0.1:58416 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 04:39:11.410647 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 04:39:11.409644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.12.15.253:38337)
0806 04:39:11.409869 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.409873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.409891 (+ 18us) server_negotiation.cc:407] Connection header received
0806 04:39:11.409948 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.409951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.410002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.410085 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 04:39:11.410522 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58420: BlockingRecv error: recv got EOF from 127.0.0.1:58420 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
I20260806 04:39:11.661058 14914 catalog_manager.cc:1259] Loaded cluster ID: 5c4028998e844b148c614d74727166cd
I20260806 04:39:11.661137 14914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: loading cluster ID for follower catalog manager: success
I20260806 04:39:11.662837 14914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: acquiring CA information for follower catalog manager: success
I20260806 04:39:11.664101 14914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be453b829cfa4103b27774fb4b554cca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:39:11.707345 14945 catalog_manager.cc:1259] Loaded cluster ID: 5c4028998e844b148c614d74727166cd
I20260806 04:39:11.707440 14945 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: loading cluster ID for follower catalog manager: success
I20260806 04:39:11.709221 14945 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: acquiring CA information for follower catalog manager: success
I20260806 04:39:11.710397 14945 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0762967d70b14be7979291d575819cdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 04:39:11.928617 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:11.924833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.12.15.253:38337)
0806 04:39:11.925104 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.925108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.925125 (+ 17us) server_negotiation.cc:407] Connection header received
0806 04:39:11.925210 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.925214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.925346 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.925442 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:11.925712 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.926239 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.927096 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.927323 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.927411 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:11.927735 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:11.928188 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:11.928300 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:11.928397 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 04:39:11.928457 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260806 04:39:11.935165 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:11.931551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.12.15.252:46457)
0806 04:39:11.931765 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.931769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.931821 (+ 52us) server_negotiation.cc:407] Connection header received
0806 04:39:11.931960 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.931963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.932095 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.932196 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:11.932449 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.932962 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.933812 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.934056 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.934132 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:11.934355 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:11.934776 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:11.934863 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:11.934945 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 04:39:11.935004 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
I20260806 04:39:11.942414 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:11.938074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.12.15.253:38337)
0806 04:39:11.938208 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.938213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.938665 (+ 452us) server_negotiation.cc:407] Connection header received
0806 04:39:11.938866 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.938870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.938927 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.938975 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:11.939302 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.940050 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.941338 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.941594 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.941690 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:11.941928 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:11.942026 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:11.942097 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:11.942198 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 04:39:11.942256 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 04:39:11.942763 15072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:11.938032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.12.15.254:32973)
0806 04:39:11.938288 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.938293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.938587 (+ 294us) server_negotiation.cc:407] Connection header received
0806 04:39:11.938834 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.938837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.939020 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.939104 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:11.939506 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.940271 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.941148 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.941455 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.941554 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:11.941914 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:11.942266 (+ 352us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:11.942345 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:11.942540 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 04:39:11.942618 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":90,"threads_started":1}
I20260806 04:39:11.943423 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 04:39:11.938746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.12.15.252:46457)
0806 04:39:11.938854 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 04:39:11.938858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 04:39:11.938934 (+ 76us) server_negotiation.cc:407] Connection header received
0806 04:39:11.939038 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 04:39:11.939041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 04:39:11.939106 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 04:39:11.939174 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 04:39:11.940406 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.941182 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.942186 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 04:39:11.942534 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 04:39:11.942607 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 04:39:11.943004 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 04:39:11.943084 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 04:39:11.943156 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 04:39:11.943253 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 04:39:11.943309 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 04:39:11.949810 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14549
I20260806 04:39:11.965356 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14661
I20260806 04:39:11.977027 12351 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5_1OQt/build/debug/bin/kudu with pid 14770
2026-08-06T04:39:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3275 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6496 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292939 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST